Subject: [arch-d] Comments on draft-iab-protocol-maintenance-03

From: S Moonesamy <[email-masked]>
Date: Wed, 08 May 2019 18:10:03 -0700

Hi Martin,

I have a few comments on draft-iab-protocol-maintenance-03.  I like 
the draft as it offers a challenging view of a principle stated by 
the IAB.  I do not agree to its argument against the robustness principle.

How can one determine whether critical details have been omitted 
(Section 3) when the protocol designer interacts with the 
implementer?  The specification may be clear to the reviewer as what 
he/she considers as obvious might be viewed as a critical detail by 
someone who does not regularly follow IETF discussions.  The 
problematic parts of a protocol can only be identified through 
feedback.  In the IETF, this is usually done through an 
erratum.  According to one of the examples given in the draft the 
protocol maintenance took eight years.  Why weren't the specification 
shortcomings addressed within a few years after publication?

Fatal conditions (Section 3) can cause a product failure.  One of the 
issues is getting two or more parties to agree on who is responsible 
for fixing the fatal condition.  It is simpler to reduce the 
likelihood of a failure in comparison with convincing the protocol 
designer or the implementer to fix the issue.

It is difficult to get approval for a protocol change (Section 4) as 
there is very little interest in revisiting the specification once it 
has been published.

The draft mentions the ecosystem (Section 4).  Is there a 
relationship between interoperability and competition [1][2] in the ecosystem?

What would be the impact if every piece of software had telemetry 
built in (Section 7.1)?  Shouldn't those errors be caught during 
testing instead of the software collecting information globally?

As an overall comment, the protocol examples in the draft are mainly 
web related.  How does protocol maintenance work in other areas?  Was 
interoperability [3] more of an advantage instead of a disadvantage?

Regards,
S. Moonesamy

1. 
https://warwick.ac.uk/fac/soc/economics/research/workingpapers/2010/twerp_924.pdf
2. http://ec.europa.eu/competition/publications/reports/kd0419345enn.pdf
3. 
https://blog.mozilla.org/netpolicy/files/2018/08/Mozilla-FTC-filing-8- 
20-2018.pdf

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 09 May 2019 02:32:21 -0400

Hey SM,

On Thu, May 9, 2019, at 11:11, S Moonesamy wrote:
> someone who does not regularly follow IETF discussions.  

I don't think there is any real science here.  People can try to talk to each other.  As you say, you need feedback to identify those things.  Yes, some things seem obvious to some and not others, but a conversation can go a long way to understanding and reconciling these little miscommunications.

See https://github.com/quicwg/base-drafts/issues/2671 for a recent example of how obvious isn't always very obvious.  

> According to one of the examples given in the draft the 

Mostly because no one bothered for a long time.  There was no community and no maintenance for many years.  When a group of people finally decided to fix the rotten thing, the pile of work was massive and the deployed base crufty.  We're talking HTTP/1.1 here, which is a big protocol with lots of woolly bits.  It has more woolly bits thanks to that period of neglect, but the community is slowly and carefully clawing away the worst of the rot and I think that good progress has been made in the past 10 years.  It's slow and deliberate, but I think that's the right approach.

> designer or the implementer to fix the issue.

Yes it is indeed simpler.  But that's what builds up the bad mojo and so I would have us reject that notion as a first option.  FWIW, I've found it a fairly pleasant experience dealing with people on the other end of protocols.  If we agree that there is a bug (which is often the case), it gets fixed.  I've had a few cases where it didn't get fixed very fast, but we tend to work out a plan for managing the problem.  Things usually work out better in the end that way.

> there is very little interest in revisiting the specification once it 

Yes.  That's a huge problem.  That's why I'm a big fan of efforts from the IESG to smooth the path for updates (draft-roach-bis-documents, the work on finding referenceable interoperable targets, and the associated work).  Institutionally, the IETF is not very well suited to this.  Despite that, we've been able to make the process work in a few groups.

> relationship between interoperability and competition [1][2] in the ecosystem?

Yes, but I really didn't want to get into that.  To a first approximation, I find that the desire for interoperability is the overriding concern in most cases.  However, we routinely get into situations where competitive pressures might affect the process of reaching an amicable conclusion.  That's why we have the IETF and other such places, and the process

> built in (Section 7.1)?

Probably a whole lot of better-informed developers and a whole lot of software that is built for use cases that matter.  There's a privacy risk, of course, but I know that Mozilla wouldn't survive without telemetry.  We have a little bit, but it is never enough.

> Shouldn't those errors be caught during 

Testing is awesome, but tests are fallible just like specifications.  Test the wrong thing and you get the wrong behaviour.  And it's possible that tests miss something entirely.  Telemetry can give you insight into problems if you ask the right questions (a difficult skill, to be sure), and provide a useful barometer for transient problems.  I doubt that you would find any server operator out there who could operate without a feed of CPU and network utilization from server instances.  What is measured differs depending on what your system does, but it's all the same principle: the longer and looser the feedback loop, the harder the system is to operate.

> interoperability [3] more of an advantage instead of a disadvantage?

Nah, this is mostly just my own bias.  As others have pointed out, we don't know if that's a problem.  The next version will probably have some sort of caveat on it to that effect.  There are lots of good examples out there, but it's hard finding concise descriptions of them.  (They tend to read like Paul Wouters' tale of IPsec woe.)

I don't quite know what you refer to in Mozilla's FTC submission there.  There's a lot going on.

---

From: S Moonesamy <[email-masked]>
Date: Thu, 09 May 2019 04:09:20 -0700

Hi Martin,
At 11:32 PM 08-05-2019, Martin Thomson wrote:
>I don't think there is any real science here.  People can try to 
>talk to each other.  As you say, you need feedback to identify those 
>things.  Yes, some things seem obvious to some and not others, but a 
>conversation can go a long way to understanding and reconciling 
>these little miscommunications.
>
>See https://github.com/quicwg/base-drafts/issues/2671 for a recent 
>example of how obvious isn't always very obvious.

Thanks for pointing me to an example.

I'll illustrate the point as follows: we are talking with each other 
as we both know where to have the discussion and with whom.  That is 
not obvious to a person who has not participated in the development 
of a protocol.  In essence, the document would have to stand on its 
own if we are interested in document clarity.  There is a thread 
about an erratum: 
https://mailarchive.ietf.org/arch/msg/yam/MtuLDdVBbG4U2Schf-xDEwsXAt0 
which might illustrate that point.

>Mostly because no one bothered for a long time.  There was no 
>community and no maintenance for many years.  When a group of people 
>finally decided to fix the rotten thing, the pile of work was 
>massive and the deployed base crufty.  We're talking HTTP/1.1 here, 
>which is a big protocol with lots of woolly bits.  It has more 
>woolly bits thanks to that period of neglect, but the community is 
>slowly and carefully clawing away the worst of the rot and I think 
>that good progress has been made in the past 10 years.  It's slow 
>and deliberate, but I think that's the right approach.

The IETF used to be discourage efforts about protocol maintenance 
(please see the old discussions about maturity levels).  As you 
mentioned, it is only when there is a significant push that the 
specification goes through another review and that is far from easy 
when the specification is a decade old.

>Yes it is indeed simpler.  But that's what builds up the bad mojo 
>and so I would have us reject that notion as a first option.  FWIW, 
>I've found it a fairly pleasant experience dealing with people on 
>the other end of protocols.  If we agree that there is a bug (which 
>is often the case), it gets fixed.  I've had a few cases where it 
>didn't get fixed very fast, but we tend to work out a plan for 
>managing the problem.  Things usually work out better in the end that way.

Ok.

>Yes.  That's a huge problem.  That's why I'm a big fan of efforts 
>from the IESG to smooth the path for updates 
>(draft-roach-bis-documents, the work on finding referenceable 
>interoperable targets, and the associated work).  Institutionally, 
>the IETF is not very well suited to this.  Despite that, we've been 
>able to make the process work in a few groups.

Please see the comment about maturity levels.  I agree with your 
comment about the IETF not being well suited for "updates".  There 
has been some progress in the recent past on putting a little more 
effort after the publication of the initial specification.

>Yes, but I really didn't want to get into that.  To a first 
>approximation, I find that the desire for interoperability is the 
>overriding concern in most cases.  However, we routinely get into 
>situations where competitive pressures might affect the process of 
>reaching an amicable conclusion.  That's why we have the IETF and 
>other such places, and the process

Ok.

>Probably a whole lot of better-informed developers and a whole lot 
>of software that is built for use cases that matter.  There's a 
>privacy risk, of course, but I know that Mozilla wouldn't survive 
>without telemetry.  We have a little bit, but it is never enough.

I am not keen on using telemetry (excluding the Mozilla case) as one 
has to do a PIA.  There was a recent case about telemetry: 
https://www.rijksoverheid.nl/documenten/rapporten/2018/11/07/data-protection-impact-assessment-op-microsoft-office

>Testing is awesome, but tests are fallible just like 
>specifications.  Test the wrong thing and you get the wrong 
>behaviour.  And it's possible that tests miss something 
>entirely.  Telemetry can give you insight into problems if you ask 
>the right questions (a difficult skill, to be sure), and provide a 
>useful barometer for transient problems.  I doubt that you would 
>find any server operator out there who could operate without a feed 
>of CPU and network utilization from server instances.  What is 
>measured differs depending on what your system does, but it's all 
>the same principle: the longer and looser the feedback loop, the 
>harder the system is to operate.

Ok.

>Nah, this is mostly just my own bias.  As others have pointed out, 
>we don't know if that's a problem.  The next version will probably 
>have some sort of caveat on it to that effect.  There are lots of 
>good examples out there, but it's hard finding concise descriptions 
>of them.  (They tend to read like Paul Wouters' tale of IPsec woe.)
>

There was a comment in that submission about interoperability.

Regards,
S. Moonesamy

---

From: Guntur Wiseno Putra <[email-masked]>
Date: Thu, 9 May 2019 21:02:36 +0700

Dear All,

Is there ones like these at either www.iab.org or  www.ietf.org...?

"A Cybermap on Protocols (Development):
It is supposedly helpful for newcomers to get toward such a knowledge on
how the Internet protocols collaborate currently --may it be better if
there are also cybermaps working on historical collaborations of
protocols...

To get into the concepts, please see:

www.cybergeography.org on "conceptual" or

https://personalpages.manchester.ac.uk/staff/m.dodge/cybergeography/atlas/conceptual.html

That supposedly concern with what of the previous message sent by S.
Moonesamy

I'll illustrate the point as follows: we are talking with each other as we
both know where to have the discussion and with whom.  That is not obvious
to a person who has not participated in the development of a protocol.  In
essence, the document would have to stand on its own if we are interested
in document clarity.  There is a thread about an erratum:
https://mailarchive.ietf.org/arch/msg/yam/MtuLDdVBbG4U2Schf-xDEwsXAt0 which
might illustrate that point.

Regard,
Guntur Wiseno Putra

Pada Kamis, 09 Mei 2019, S Moonesamy <[email-masked]> menulis:

>> I don't think there is any real science here.  People can try to talk to
>> each other.  As you say, you need feedback to identify those things.  Yes,
>> some things seem obvious to some and not others, but a conversation can go
>> a long way to understanding and reconciling these little miscommunications.
>>
> I'll illustrate the point as follows: we are talking with each other as we
> both know where to have the discussion and with whom.  That is not obvious
> to a person who has not participated in the development of a protocol.  In
> essence, the document would have to stand on its own if we are interested
> in document clarity.  There is a thread about an erratum:
> Mostly because no one bothered for a long time.  There was no community
>> and no maintenance for many years.  When a group of people finally decided
>> to fix the rotten thing, the pile of work was massive and the deployed base
>> crufty.  We're talking HTTP/1.1 here, which is a big protocol with lots of
>> woolly bits.  It has more woolly bits thanks to that period of neglect, but
>> the community is slowly and carefully clawing away the worst of the rot and
>> I think that good progress has been made in the past 10 years.  It's slow
> The IETF used to be discourage efforts about protocol maintenance (please
> see the old discussions about maturity levels).  As you mentioned, it is
> only when there is a significant push that the specification goes through
> another review and that is far from easy when the specification is a decade
> old.
>
> Yes it is indeed simpler.  But that's what builds up the bad mojo and so I
>> would have us reject that notion as a first option.  FWIW, I've found it a
>> fairly pleasant experience dealing with people on the other end of
>> protocols.  If we agree that there is a bug (which is often the case), it
>> gets fixed.  I've had a few cases where it didn't get fixed very fast, but
>> we tend to work out a plan for managing the problem.  Things usually work
>> out better in the end that way.
>>
>
> Ok.
>
> Yes.  That's a huge problem.  That's why I'm a big fan of efforts from the
>> IESG to smooth the path for updates (draft-roach-bis-documents, the work on
>> finding referenceable interoperable targets, and the associated work).
>> Institutionally, the IETF is not very well suited to this.  Despite that,
>> we've been able to make the process work in a few groups.
>>
>
> Please see the comment about maturity levels.  I agree with your comment
> about the IETF not being well suited for "updates".  There has been some
> progress in the recent past on putting a little more effort after the
> publication of the initial specification.
>
> Yes, but I really didn't want to get into that.  To a first approximation,
>> I find that the desire for interoperability is the overriding concern in
>> most cases.  However, we routinely get into situations where competitive
>> pressures might affect the process of reaching an amicable conclusion.
>> That's why we have the IETF and other such places, and the process
>>
>
> Ok.
>
> Probably a whole lot of better-informed developers and a whole lot of
>> software that is built for use cases that matter.  There's a privacy risk,
>> of course, but I know that Mozilla wouldn't survive without telemetry.  We
>> have a little bit, but it is never enough.
>>
>
> I am not keen on using telemetry (excluding the Mozilla case) as one has
> to do a PIA.  There was a recent case about telemetry:
> https://www.rijksoverheid.nl/documenten/rapporten/2018/11/07
> /data-protection-impact-assessment-op-microsoft-office
>
> Testing is awesome, but tests are fallible just like specifications.  Test
>> the wrong thing and you get the wrong behaviour.  And it's possible that
>> tests miss something entirely.  Telemetry can give you insight into
>> problems if you ask the right questions (a difficult skill, to be sure),
>> and provide a useful barometer for transient problems.  I doubt that you
>> would find any server operator out there who could operate without a feed
>> of CPU and network utilization from server instances.  What is measured
>> differs depending on what your system does, but it's all the same
>> principle: the longer and looser the feedback loop, the harder the system
>> is to operate.
>>
>
> Ok.
>
> Nah, this is mostly just my own bias.  As others have pointed out, we
>> don't know if that's a problem.  The next version will probably have some
>> sort of caveat on it to that effect.  There are lots of good examples out
>> there, but it's hard finding concise descriptions of them.  (They tend to
>> read like Paul Wouters' tale of IPsec woe.)
>>
> _______________________________________________
> Architecture-discuss mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/architecture-discuss
>

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 09 May 2019 20:42:17 -0400

> which might illustrate that point.

Yeah, that's a good example.  Everyone agrees that the spec is sub-optimal, but we have an overwhelming agreement that fixing it isn't what happens next.

But the conversation happened, and that's not so bad.
 
> https://www.rijksoverheid.nl/documenten/rapporten/2018/11/07/data-protection-impact-assessment-op-microsoft-office

We might have to disagree on that point.  There are very interesting questions about who gets to decide what is OK to collect, and how we manage the concerns of those involved.  Steve Bellovin just gave a great talk on the failure of "notice and consent" to the IAB that I hope we'll be able to get in front of the community; that's clearly not the answer.  But we're veering well off into the weeds when we get there.
 
> There was a comment in that submission about interoperability.

Yes, there was.  Several.

---

From: Ted Hardie <[email-masked]>
Date: Fri, 10 May 2019 09:24:59 -0700

On Thu, May 9, 2019 at 5:42 PM Martin Thomson <[email-masked]> wrote:

> We might have to disagree on that point.  There are very interesting
> questions about who gets to decide what is OK to collect, and how we manage
> the concerns of those involved.  Steve Bellovin just gave a great talk on
> the failure of "notice and consent" to the IAB that I hope we'll be able to
> get in front of the community; that's clearly not the answer.

It's now up on Steve's website:

https://www.cs.columbia.edu/~smb/talks/iab-privacy.pdf

>   But we're veering well off into the weeds when we get there.
>

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 10 May 2019 13:53:46 -0400

One interesting thought in the slides related to the discussion,
paraphrased: high "surface" variability in protocols and their
implementation provides tracking metadata, making it easier to identify
users even if they are not identified explicitly. In other words, if every
implementation sticks to the same wire format, without non-semantic
variability, it's a lot harder to track people. I know that there are
companies that use SIP signaling variability to identify originating
carriers. (In that case, it seems to be helpful for fraud detection.)

Henning