Subject: [Tsv-art] Slides and Notes from TSV-ART meeting 180318

From: Magnus Westerlund <[email-masked]>
Date: Mon, 19 Mar 2018 11:32:29 +0000

Hi,

The TSV-ART meet yesterday and went through the attached slides. There 
was good feedback and discussion. Those not present, please review them 
and send any questions or additional feedback to the tsv-art mailing list.

We discussed that the Triage team should ensure that reminders goes out 
after 24 hours (tool question) and failure to respond in two days 
(Working days) should result in triage team to reassign the review.

We should ensure that the wiki is no longer using pages called TSV 
Directorate. Likely need to migrate text to new pages and put in 
pointers in the old ones.

The reviewers are requested to put in reviews early. Also a quick review 
from you can be good for several reasons. It can identify something that 
needs more review, or work as an indication to the AD that they need to 
defer the document to a future telechat. Also, if you are near the 
deadline, but will be a little late send a message that it is coming so 
the ADs can hold of balloting until your review arrives.

Colin noted that as the tracker sends email with a source address with 
your registered address the review sent by the tracker has a tendency to 
be caught by spam filters as they don't have the right headers. So it 
can actually be a recommendation to send mail manually and link the 
review after you sent it.

There was also a number of tool questions.

- Automatically send reminders on request for reviewing

- The field for writing in review should be pre-populated with the 
boiler plate text for reviews.

- Request that source email used is one that will not be as easily blocked.

- We will need to check which email address a review sent by data 
tracker goes to when the class is early review.

Cheers

Magnus Westerlund

----------------------------------------------------------------------
Network Architecture & Protocols, Ericsson Research
----------------------------------------------------------------------
Ericsson AB                 | Phone  +46 10 7148287
Torshamnsgatan 23           | Mobile +46 73 0949079
SE-164 80 Stockholm, Sweden | mailto: [email-masked]
----------------------------------------------------------------------

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Mar 2018 07:16:54 -0700

Hi, all,

My only concern is the expectation of a 2-working-day turnaround.

Nothing in the IETF needs to have even an ACK on that timescale. 4-5 days (i.e., one week) should be more than sufficient. 

Joe

> On Mar 19, 2018, at 4:32 AM, Magnus Westerlund <[email-masked]> wrote:
> 
> Hi,
> 
> The TSV-ART meet yesterday and went through the attached slides. There was good feedback and discussion. Those not present, please review them and send any questions or additional feedback to the tsv-art mailing list.
> 
> We discussed that the Triage team should ensure that reminders goes out after 24 hours (tool question) and failure to respond in two days (Working days) should result in triage team to reassign the review.
> 
> We should ensure that the wiki is no longer using pages called TSV Directorate. Likely need to migrate text to new pages and put in pointers in the old ones.
> 
> The reviewers are requested to put in reviews early. Also a quick review from you can be good for several reasons. It can identify something that needs more review, or work as an indication to the AD that they need to defer the document to a future telechat. Also, if you are near the deadline, but will be a little late send a message that it is coming so the ADs can hold of balloting until your review arrives.
> 
> Colin noted that as the tracker sends email with a source address with your registered address the review sent by the tracker has a tendency to be caught by spam filters as they don't have the right headers. So it can actually be a recommendation to send mail manually and link the review after you sent it.
> 
> - The field for writing in review should be pre-populated with the boiler plate text for reviews.
> 
> - We will need to check which email address a review sent by data tracker goes to when the class is early review.
> 
> Cheers
> 
> Magnus Westerlund
> 
> <20180314-01-TSV-ART-Review-Tutorial.pptx>_______________________________________________
> Tsv-art mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tsv-art

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 19 Mar 2018 14:43:16 +0000

> Nothing in the IETF needs to have even an ACK on that timescale. 4-5 days (i.e., one week) should be more than sufficient.

For IETF last calls and IESG agenda reviews the time from awareness to 
deadline is usually 2 weeks, and unfortunately in many case for an IESG 
review it may be more like 1-week. Thus, if we are going to reassign it 
if a reviewer rejects it a 4-5 day time scale would result in that there 
might be as little as 1-2 days to preform the review. Thus, if you can't 
respond to the email within 2 working days, then I rather reassign a 
review, than waiting. But, maybe 3 days we could stretch to, but I don't 
think more to allow for anyone taking over the review.

Cheers

Magnus Westerlund

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Mon, 19 Mar 2018 14:53:39 +0000

It’s just clicking a button as soon as you see an email. My assumption was that people check their mails at least every two days in a „normal“ working situation.

Mirja

> Am 19.03.2018 um 14:43 schrieb Magnus Westerlund <[email-masked]>:
> 
> 
> 
> For IETF last calls and IESG agenda reviews the time from awareness to deadline is usually 2 weeks, and unfortunately in many case for an IESG review it may be more like 1-week. Thus, if we are going to reassign it if a reviewer rejects it a 4-5 day time scale would result in that there might be as little as 1-2 days to preform the review. Thus, if you can't respond to the email within 2 working days, then I rather reassign a review, than waiting. But, maybe 3 days we could stretch to, but I don't think more to allow for anyone taking over the review.
> 
> Cheers
> 
> Magnus Westerlund
> 
> _______________________________________________
> Tsv-art mailing list
> [email-masked]

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Mar 2018 07:56:09 -0700

Shout we have much better lead times than needing to jump just because of last calls or agendas?

I know I’ve been party to debates with much better leads that turned into “urgent requests” last minute due to poor planning (a non TVs one sits in my inbox today despite participating int the discussion over a month ago).

I don’t think poor planning is a good justification for pushing that burden on us.

Joe

> On Mar 19, 2018, at 7:43 AM, Magnus Westerlund <[email-masked]> wrote:
> 
> 
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Mar 2018 08:02:20 -0700

See email - yes.

Able to commit and schedule resources to know the timeline can be met - not necessarily. 

I.e., you’re not just asking “are you there” but “can you review this in time”.

Joe

> On Mar 19, 2018, at 7:53 AM, Mirja Kuehlewind (IETF) <[email-masked]> wrote:
>

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 19 Mar 2018 15:17:28 +0000

Joe and all,

> Shout we have much better lead times than needing to jump just because of

Last Calls are either two or four weeks, and that's probably what we can
achieve with the current IETF, as far as earlier notice.

I do point out that we have three new ADs and we really CAN tell the IESG
how much lead time our reviewers think is reasonable for telechat reviews
before the corresponding telechats. This is the time to change the way the
IESG does IESG Evaluation.

If we make progress on that, we can revisit getting more lead time for Last
Call reviews.

We didn't talk about it during the meeting, but would the review team be OK
with getting requests going to the triage team at completion of WGLC?
That's probably the first state transition that would give you folks more
notice.

Spencer, who was REALLY impressed with the discussion that took place, and
is REALLY impressed with the current level of service Mirja and I are
seeing from both the triage team and the reviewers. So, thank you all for
that.

---

From: "Mirja Kuehlewind (IETF)" <[email-masked]>
Date: Mon, 19 Mar 2018 16:28:56 +0000

Hi Joe,

this was not seen as problem by the people in the room. Usually the load per person is really low (maybe 1 draft per 6 months) and my assumption was that the default is that you will be able to squeeze it in. Especially as most requests will actually have a deadline that is usually 2 weeks to go. If you are in a more than usually loaded situation or the deadline is really tight and you are not sure about your availability, it might actually be better to decline the review quickly such that we can quickly re-assign it to someone else. The 2 days deadline is really more to cover the case (and converge quickly) where someone is not checking emails at all (e.g. due to holidays or other „issues“ that the triage team did not know about when the assignment was made). However, in case of doubt, if you saw and read the mail but you don’t know what to reply yet, sending a (manual) email to the triage team and letting them know that you are aware of the assignment is probably the best thing to do. Would that work for you?

Mirja

> Am 19.03.2018 um 15:02 schrieb Joe Touch <[email-masked]>:
> 
> See email - yes.
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 19 Mar 2018 18:38:29 -0700

On 3/19/2018 9:28 AM, Mirja Kuehlewind (IETF) wrote:

> However, in case of doubt, if you saw and read the mail but you don’t know what to reply yet, sending a (manual) email to the triage team and letting them know that you are aware of the assignment is probably the best thing to do. Would that work for you?

That works., but  my feedback is that the IETF should not work in a very
casual mode most of the year while expecting near-realtime interaction
at their convenience (e.g., when issuing last calls, IESG reviews, or
during IETF meetings).

Spencer has the right approach, IMO - pushback that sort of behavior:

> I do point out that we have three new ADs and we really CAN tell the
> IESG how much lead time our reviewers think is reasonable for telechat
> reviews before the corresponding telechats. This is the time to change
> the way the IESG does IESG Evaluation.

Joe