Subject: [Mtgvenue] Venue 'requirements' language

From: Dave Crocker <[email-masked]>
Date: Fri, 28 Oct 2016 13:25:14 +0200

Folks,

I'd like some help on the document's normative vocabulary.  The current 
text is:

> Mandatory:
>     If this requirement cannot be met, a location under consideration is unacceptable. We walk away.
> Desired:
>     We would very much like to meet this requirement, but have frequently been unable to. The fact that we could not meet it is considered in comparison to other sites.
> Important:
>     Can be a make-or-break consideration, but can also be traded off against other considerations.
> Would be nice:
>     Not make-or-break, but warrants additional consideration if found to be true.

As basic vocabulary choices I have been finding especially 'desired' vs. 
'important' to be confusing.

Worse, for 'important' it is not obvious what tradeoffs are permitted or 
excluded.  And while any one of us might formulate a reasonable answer, 
each of us would do it differently and thereby produce inconsistent 
results.  Since my understanding of this documentation exercise is that 
it seeks to improve consonance between community preferences and meeting 
venue selection choices, inconsistency from one person to the next is 
strongly contraindicated.

With respect to specific vocabulary, I suggest some small changes:

> Mandatory:
>     If this requirement cannot be met, a location under consideration is unacceptable. We walk away.
> Important:
>     We would very much like to meet this requirement, but have frequently been unable to. The fact that we could not meet it is considered in comparison to other sites.
> Tradeoff:
>     Can be a make-or-break consideration, but can also be traded off against other considerations.
> Desired:
>     Not make-or-break, but is should get active effort to satisfy

(Note that this demotes 'desired' to the 'would be nice' definition and 
that the associated definition is slightly different from its original.)

As for Tradeoffs, we ought to consider how it gets used, in practical 
terms.  The challenge is to avoid over-specifying.  For example a 
natural suggestion might be to list the items that can be traded off, 
but first we probably can't predict the set adequately and second it 
would produce an overly complicated model.

On reflection, perhaps we merely should use the existing definition and 
specify that the selection process : a) will note when tradeoff items 
are at issue, and b) will document what the basis for the traded-off 
choice(s) was?

Thoughts?

d/

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Alissa Cooper <[email-masked]>
Date: Fri, 28 Oct 2016 14:42:52 -0400

I wonder if four levels is over-complicating this. Perhaps mandatory and desired would be sufficient to capture the reality of venue selection.

Alissa

> On Oct 28, 2016, at 7:25 AM, Dave Crocker <[email-masked]> wrote:
> 
> Folks,
> 
> I'd like some help on the document's normative vocabulary.  The current text is:
> 
>> Mandatory:
> As basic vocabulary choices I have been finding especially 'desired' vs. 'important' to be confusing.
> 
> Worse, for 'important' it is not obvious what tradeoffs are permitted or excluded.  And while any one of us might formulate a reasonable answer, each of us would do it differently and thereby produce inconsistent results.  Since my understanding of this documentation exercise is that it seeks to improve consonance between community preferences and meeting venue selection choices, inconsistency from one person to the next is strongly contraindicated.
> 
> (Note that this demotes 'desired' to the 'would be nice' definition and that the associated definition is slightly different from its original.)
> 
> As for Tradeoffs, we ought to consider how it gets used, in practical terms.  The challenge is to avoid over-specifying.  For example a natural suggestion might be to list the items that can be traded off, but first we probably can't predict the set adequately and second it would produce an overly complicated model.
> 
> On reflection, perhaps we merely should use the existing definition and specify that the selection process : a) will note when tradeoff items are at issue, and b) will document what the basis for the traded-off choice(s) was?
> 
> Thoughts?
> 
> d/
> 
> -- 
> 
>  Dave Crocker
> _______________________________________________
> Mtgvenue mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/mtgvenue

---

From: Dave Crocker <[email-masked]>
Date: Fri, 28 Oct 2016 21:00:50 +0200

> I wonder if four levels is over-complicating this. Perhaps mandatory and desired would be sufficient to capture the reality of venue selection.

That's what we originally had.  Somewhere along the way there was a 
desire to get more nuance.  I can imagine it could be operationally 
workable, but feel that we need to do more up-front work to make 
relatively explicit how.

d/

-- 

   Dave Crocker

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 29 Oct 2016 08:57:26 +1300

> workable, but feel that we need to do more up-front work to make 

I find 'mandatory' troubling, too. Let's say we resolve the one we discussed
yesterday as Mandatory: the hotel must guarantee a block of N rooms.
(Regardless of how N is determined). But a hotel can only guarantee N-1 rooms,
and is otherwise perfect. What do we do?

So, I wonder if we shouldn't just have 'Important' and 'Desirable'.

   Brian

---

From: Dave Crocker <[email-masked]>
Date: Mon, 31 Oct 2016 08:24:47 -0700

> So, I wonder if we shouldn't just have 'Important' and 'Desirable'.

With respect to process issues in the IETF, we have a remarkable fear of 
being specific and firm, in marked contrast to the details we tend to 
specify in our technical documents.

Some of this is a very reasonable and constructive desire to make sure 
that delegated tasks really are delegated.  When we specify too much 
detail, we have removed the ability of staff to make judgements.

But the other side of this is that we suffer the danger of not providing 
staff with enough guidance about what the community does require. 
Worse, we've made it far too easy for the community to be fickle.  So 
then we get to surprise them, post hoc, with criticisms that they did 
not satisfy our requirements (which we never adequately communicated...)

I think our challenge is to figure out what issues are basic, community 
requirements/desires and what issues should be for the staff to decide.

With the topic of primary hotel as the current example, I think the 
community issue is:  should their should be a 'primary' hotel, in the 
established sense, seems to me a community issue? (The arguments in 
favor are along the lines in the other note I just published.)  Just how 
many rooms should be in the contracted block seems to me something that 
should be left to staff assessment.

d/
-- 

   Dave Crocker

---

From: Dave Crocker <[email-masked]>
Date: Mon, 31 Oct 2016 08:43:56 -0700

> I wonder if four levels is over-complicating this. Perhaps mandatory and desired would be sufficient to capture the reality of venue selection.

After now wandering through the draft several more times, I think that 
there's an essential point that the mandatory vs. important labeling 
distinction is trying to highlight, but I also think that we chose the 
wrong way to pursue it.

There are some things that are not negotiable. If we can't satisfy them, 
we don't meet at that venue.  We state these as simple, direct and 
mandatory.

There are some other things that also can be described with simple and 
direct language, but those things are not mandatory.  If we can't 
satisfy them, it's unfortunate, but we can still meet at the venue. 
Such items factor into comparisons between venue choices that the staff 
makes, but do not factor beyond such comparisons.

However there are other things that have some flexibility, but 'some 
flexibility' is not the same as saying that they are 'optional/desired'. 
  That is, there is a real and basic requirement that /must/ be met, but 
the community is willing to have the details vary.

I think that the solution to this is not by special labeling but rather 
by special phrasing of the requirement.  In effect, the requirement 
needs to be phrased more like a process requirement than as a simple, 
objective criterion.

Looking at the "City Criteria" section's list of items classed as 
"important" I believe that what we mean is that it is /mandatory/ that 
these issues be considered and that it is mandatory that the community 
find the choices acceptable.  So any difficulties or exceptions to be 
negotiated with the community.

Each of the 'Important' items in the City Venue list is inherently 
squishy but each also is essential.  Squishy in the sense that there 
they are not subject to a simple quantification, but essential in that 
the community /must/ be satisfied with the choice.

So, for example, "Travel to the venue is reasonably acceptable based on 
cost, time, and burden for participants traveling from multiple 
regions."  I believe that this phrasing accurately specifies a 
requirement.  That is, it is not negotiable.  There is no trade-off. 
The community /must/ find the details acceptable.  What is negotiable is 
what the specific details are.

For efficiency, there's a common set of choices that have satisfied the 
community and that the staff can therefore use without much worry about 
community acceptance.  But when the possibility of exceptional details 
surfaces, there is nothing wrong with going outside of the established 
norm...  as long as the community deems the exception (or perhaps 
long-term change) acceptable.

So the wording for such requirements needs to target community 
acceptance of the details, without defining what those details are.  The 
text, therefore, leaves those details to the staff processes, including 
consulting the community when needed.

Thoughts?

d/

-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 31 Oct 2016 13:46:31 -0400

On Mon, Oct 31, 2016 at 08:43:56AM -0700, Dave Crocker wrote:
> special phrasing of the requirement.  In effect, the requirement needs to be
> phrased more like a process requirement than as a simple, objective
> criterion.

> Thoughts?

I think this is the opposite direction to the draft that I put
together with Alissa, but I think it might work.

The reason I, at least, worked on the draft we submitted was that I
felt draft-baker-mtgvenue-iaoc-venue-selection-process had two issues:
too many things listed, and no guidance for the IAOC to understand
when it had done the right thing.

This "do the right thing" problem is I think part of the issue we've
faced.  The IAOC knows there are a bunch of things to take into
account.  What it does not know is how to balance the various things
against each other, and as a result it sometimes just blows it and
everyone gets angry.

I thought that a way to answer that was to give an ordered list of
priorities.  This would mean that a venue that satisfied them all
would be preferable to one that didn't, but if the IAOC couldn't get
everything then it would be in a position to tell the community, "We
tried, we couldn't get everything, and here's the way we decided
(which has consensus)."

An alternative is, I suppose, to list the criteria that must somehow
be satisfied and to describe what satisfaction looks like without
specifying the particular satisfaction mechanism.  The problem with
that, of course, is that it puts the squishiness issue back onto the
IAOC and makes way for these sorts of dispute in the future, too.  But
at least there's an unambiguous list of what must be achieved.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Mon, 31 Oct 2016 11:12:54 -0700

On 10/31/2016 10:46 AM, Andrew Sullivan wrote:
> priorities.

Ahh.  Sorry.  I, for one, managed to miss that this was meant as an 
ordered list.

While I laud the intent, alas, I don't think the community has anything 
approaching a priority-ranked model of such issues.  Not at any one 
moment in time, and certainly not stable across time.

I think we can reasonably list a set of things that are 
important-but-imprecise.  (I find myself thinking of them as subject to 
'clinical' assessments.) I think we can develop a sense of "normal" 
choices for these and maybe a sense of "reasonable" tradeoffs within 
that normal.

And after that I think the best we can do is have staff formulate 
proposals and seek counsel from the community.

The real contribution of the list, then, would be to flag the areas of 
concern that require this sort of clinical processes between staff and 
community.

> This would mean that a venue that satisfied them all

I suspect we disagree, for most of these.  Some, sure.  Others, no.

If an area is deemed unsafe we don't meet there.  The squishiness is the 
assessment of safety.  If a city is too difficult to reach or too 
expensive, we don't meet there.  The squishiness in the assessments of 
difficulty and cost.  If the entry barriers are deemed to onerous, we 
don't meet there.  The squishiness...

That is, these types of issue can trade amongst themselves to a degree, 
perhaps, but with some very basic limits -- except we don't know what 
those limits are and they probably change over time. (sigh.)  Somewhat 
more remote, for somewhat less expensive, for example.  But not /too/ 
remote.

We can perhaps vary within the item, but we can't dismiss it entirely.

> at least there's an unambiguous list of what must be achieved.

Hence the model of a stable "normal" range of tradeoffs, with community 
consultation when thinking it useful to go outside that range.  This, I 
think, reflects what has been happening with some success.

My sense is that the failure was that we didn't adequately document what 
needed to be considered, rather than that the considerations were flawed.

(I think the considerations were often deemed seriously flawed, a decade 
or more ago, but that the recent track record has been notably better. 
My assessment some years ago was that we averaged a community blow up 
almost every year.  The rate is /much/ lower now...)

d/
-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 31 Oct 2016 15:23:50 -0400

Hi,

On Mon, Oct 31, 2016 at 11:12:54AM -0700, Dave Crocker wrote:
> 
> Ahh.  Sorry.  I, for one, managed to miss that this was meant as an ordered
> list.

A lot of people did.  I'm not sure how we could better have said,
"This memo offers a list of objectives, in descending order of
importance, in an attempt to guide decision-makers."  :-)

> approaching a priority-ranked model of such issues.  Not at any one moment
> in time, and certainly not stable across time.

I think my view is that we don't have such a consensus ranking now,
but that if we're going to head off these disputes in the future I
think it's the most likely way to do it.

> And after that I think the best we can do is have staff formulate proposals
> and seek counsel from the community.

But I don't think that's a real possibility.  The point that the IAOC
has made repeatedly is that you can't negotiate these things in
public.  Staff says that over and over.  So the consultation the IAOC
is already doing may really be the practical limit to what it can do.

> assessment of safety.

And "deemed".  Americans are by and large way more tolerant of risks
from gun violence than many other people are, for instance.  But in
general I agree with you.

> If a city is too difficult to reach or too expensive,
> we don't meet there.  The squishiness in the assessments of difficulty and
> cost.

And "too".  But yes.

> If the entry barriers are deemed to onerous, we don't meet there.
> The squishiness...

"Too onerous" is another one of those examples where it's either
trivially true or false.  

> We can perhaps vary within the item, but we can't dismiss it entirely.

I bet all the problem here is going to come from the marginal cases
and the dispute over what is "reasonable".

> (I think the considerations were often deemed seriously flawed, a decade or
> more ago, but that the recent track record has been notably better. My
> assessment some years ago was that we averaged a community blow up almost
> every year.  The rate is /much/ lower now...)

This may well be true.  One thing of course that we have started doing
is returning to some places repeatedly.  So there's that.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 1 Nov 2016 08:29:02 +1300

I find myself in remarkably good agreement with Dave on a couple of points:

On 01/11/2016 04:24, Dave Crocker wrote:
...
> should be left to staff assessment.

On 01/11/2016 04:43, Dave Crocker wrote:
...
> needs to be phrased more like a process requirement than as a simple, 

Right. We cannot take all the room for judgment and compromise away from
the IAOC; the IAOC is not an algorithm.

   Brian

---

From: Dave Crocker <[email-masked]>
Date: Mon, 31 Oct 2016 12:53:44 -0700

> importance, in an attempt to guide decision-makers."  :-)

Let me get this straight.  You think that a simple, straightforward, 
introductory bit of text that explains exactly what you are doing 
suffices with IETF readers?  And you've been participating for how long?

harumph.

Please remember that we have a community that has become quite adamant 
in declaring that must is different from MUST.  This should tell you 
something about how we read things. And how we don't...

> think it's the most likely way to do it.

(It's probably obvious, but it occurs to me that I should be explicit 
that I'm solidly not wearing the editor hat.  We've moved fully into 
personal opinion from me...)

I'm pretty sure that this requires far too much complexity in the model. 
  Too complex to formulate and even more complex to use.  And we aren't 
all that skilled at processing complex decision model.

Worse, I will repeat that I think it highly unlikely the model would be 
stable.  Even if we could obtain something that looks like a community 
rough consensus at a given moment, I suspect it's half-life of stability 
would be measured in hours.  If that.

> is already doing may really be the practical limit to what it can do.

First, I think the IAOC has actually been reasonably diligent about 
knowing when to bring a variety of unusual choices to the community.  (I 
have disagreements about how some of them are presented or evaluated, 
but that's quite different from whether they were properly identified.)

Rather the problem is that there are some issues that the IAOC didn't 
anticipate but some community folk /assumed/ would be considered.  So 
the challenge is to get them identified early, rather than to direct how 
they should be evaluated.

> "Too onerous" is another one of those examples where it's either

I think we are wildly inconsistent is assessing this, but by and large, 
the community seems reasonably comfortable with the choices the IAOC has 
been making, including some rejections.

> and the dispute over what is "reasonable".

Hence the model of a relatively comfortable set of norms for topics of 
concern, with a fallback to community consultation when there is doubt. 
As long as the list of concerns is reasonably well identified and 
stable, it's safe for the norms to be quite conservative, since it only 
results in some extra community queries early, rather than extra 
community blowups later...

> is returning to some places repeatedly.  So there's that.

if only we'd thought of that sooner.

oh, wait...

d/

-- 

   Dave Crocker

---

From: Andrew Sullivan <[email-masked]>
Date: Mon, 31 Oct 2016 16:08:57 -0400

(Just as Dave noted, I hope it's also obvious that I'm wearing no hat
in this discussion.)

On Mon, Oct 31, 2016 at 12:53:44PM -0700, Dave Crocker wrote:
> I think we are wildly inconsistent is assessing this, but by and large, the
> community seems reasonably comfortable with the choices the IAOC has been
> making, including some rejections.

I suspect part of the problem here comes down to what "community"
means in this context.

Part of the difficulty we may have had is that the definition of
"community" has changed over time, and we're a remarkably conservative
organization and therefore not that welcoming to people who don't fit
the majority model.  Over time, of course, the majority model changes,
but not always in ways that are what those excluded might consider
"quickly".

The IETF is in some way going to have to struggle with this, however,
because many of us are reaching the age of retirement.  The complexion
(both literally and metaphorically) of the IETF will change in the
coming years, I think, and I am not sanguine that the list we come up
with, however it is supposed to guide the IAOC, is going to be a super
stable one.

> long as the list of concerns is reasonably well identified and stable, it's
> safe for the norms to be quite conservative, since it only results in some
> extra community queries early, rather than extra community blowups later...

At the possible expense of a difficult consensus now :)  But sure.

> if only we'd thought of that sooner.

Yeah, well, don't get me started.

A

-- 
Andrew Sullivan
[email-masked]

---

From: Dave Crocker <[email-masked]>
Date: Mon, 31 Oct 2016 13:15:36 -0700

> I suspect part of the problem here comes down to what "community"

And here I was, posting a note earlier about using terms while only 
assuming their definition... heh.

I'm using 'staff' to mean the set of folk who do the behind-the-scenes 
work concerning sites.  This includes IAOC, Meetings committee, ISOC 
staff and AMS staff, and maybe a few more individuals.

I'm using 'community' to the the general IETF community, probably best 
defined in terms of the ietf mailing list.  (Yeah, that excludes too 
many folk these days, but it is nonetheless the medium for communication 
when something needs to be discussed.

> Yeah, well, don't get me started.

you?  marhsall rose and i formulated a proposal in the early 90s (but i 
can't find  copy of.)

double harumph.

d/
-- 

   Dave Crocker