Subject: [Gen-art] Gen-ART review of draft-johansson-loa-registry-04

From: <[email-masked]>
Date: Sun, 1 Apr 2012 16:57:07 -0400

I am the assigned Gen-ART reviewer for this draft. For background on 
Gen-ART, please see the FAQ at 
<http://wiki.tools.ietf.org/area/gen/trac/wiki/GenArtfaq>.

Please resolve these comments along with any other Last Call comments 
you may receive.

Document: draft-johansson-loa-registry-04
Reviewer: David L. Black
Review Date: April 1, 2012
IETF LC End Date: April 3, 2012
IESG Telechat date: April 12, 2012

This draft establishes an IETF registry of SAML Level of Assurance (LoA) profiles;
it's short and clear, although it does not contain any initial content for the
registry - presumably that will be supplied after the registry is created via the
expert review registration mechanism established by this draft. 

Summary:
This draft is on the right track but has open issues, described in the review.

Major issues: (1)
My major open issue concerns the last paragraph in the Introduction:

   Although the registry will contain URIs that reference SAML
   Authentication Context Profiles other protocols MAY use such URIs to
   represent levels of assurance definitions without relying on their
   SAML XML definitions.  Use of the registry by protocols other than
   SAML or OpenID Connect is encouraged.

While this is good in principle, and one presumes that each registration of
sets of profiles from an existing protocol will be self-consistent, this
text also encourages other (e.g., new) protocols to draw upon this registry
without providing any guidance.  I'm concerned that it's probably possible
to make a serious mess in a new protocol by using an LoA or two from multiple
sets of registered LoAs without paying attention to whether the resulting
collection of LoAs is consistent or coherent (or even sensible) for use in
a single protocol.  This concern is reinforced by the guidance to expert
reviewers in Section 4.1, which effectively conveys a desire to get all
of the reasonable LoA profiles registered here, regardless of source or
consistency with other registered LoA profiles.

I'd like to see some guidance to protocol designers and others for how to
appropriately select multiple LoA profiles from this registry in a fashion
that results in a consistent and (hopefully) usable collection.  For example,
it may be a good idea to use (or start with) a set of related profiles already
in use by an existing protocol in preference to mixing/matching individual
profiles from multiple existing protocols.  At some level, this is common
sense advice that the presence of profiles in this registry does not obviate
the need to apply good design judgment, but that does deserve to be stated.  

Minor issues: (2) 

(1) This draft is intended to be an informational RFC, but it uses
RFC 2119 keywords.  That's only a good idea in exceptional circumstances. I
suggest removing section 1.1 and replacing upper case MUST/SHOULD/MAY with
their lower case versions or reworded explanations of rationale.  Most of
the uses of RFC 2119 keywords are not protocol requirements, but requirements
on IANA, registrants, and users of the registry for which RFC 2119 keywords
are not appropriate, e.g., see RFC 2119 section 6:

   Imperatives of the type defined in this memo must be used with care
   and sparingly.  In particular, they MUST only be used where it is
   actually required for interoperation or to limit behavior which has
   potential for causing harm (e.g., limiting retransmisssions)

(2) Section 4

OLD
   The initial pool of expert and the
   review criteria are outlined below.
NEW
   The review criteria are outlined below.

The initial pool of experts is not designated by this draft.

Nits/editorial comments:

Section 3

OLD
   The following ABNF productions represent reserved values and names
NEW
   The reserved element defined by the following ABNF productions represents
   a set of reserved values and names

Section 4

   The registry is to be operated under the "Designated Expert Review"
   policy from RFC5226 [RFC5226] employing a pool of experts

Nope, the actual RFC5226 name of that well-known IANA policy is Expert
Review (or Designated Expert), see section 4.1 of RFC5226.  If that
well-known IANA policy isn't what was intended, this is a serious
open issue.

Top of p.7
   The presense of an entry in the registy MUST NOT be taken to imply
                                         ^
r ---------------------------------------/

Section 7

OLD
   An implementor of MUST NOT treat the registry as a trust framework or
NEW
   A protocol implementor MUST NOT treat the registry as a trust framework or

The minor issue about RFC 2119 keywords also applies to this text.

idnits 2.12.13 did not find any nits that need attention.

Thanks,
--David
----------------------------------------------------
David L. Black, Distinguished Engineer
EMC Corporation, 176 South St., Hopkinton, MA  01748
+1 (508) 293-7953             FAX: +1 (508) 293-7786
[email-masked]        Mobile: +1 (978) 394-7754
----------------------------------------------------

---

From: Leif Johansson <[email-masked]>
Date: Sun, 01 Apr 2012 23:59:19 +0200

On 04/01/2012 10:57 PM, [email-masked] wrote:
> I am the assigned Gen-ART reviewer for this draft. For background
> on Gen-ART, please see the FAQ at 
> Please resolve these comments along with any other Last Call
> comments you may receive.
> 
> Document: draft-johansson-loa-registry-04 Reviewer: David L. Black 
> Review Date: April 1, 2012 IETF LC End Date: April 3, 2012 IESG
> Telechat date: April 12, 2012
> 
> This draft establishes an IETF registry of SAML Level of Assurance
> (LoA) profiles; it's short and clear, although it does not contain
> any initial content for the registry - presumably that will be
> supplied after the registry is created via the expert review
> registration mechanism established by this draft.
> 
> Summary: This draft is on the right track but has open issues,
> described in the review.
> 
> Major issues: (1) My major open issue concerns the last paragraph
> in the Introduction:
> 
> Authentication Context Profiles other protocols MAY use such URIs
> to represent levels of assurance definitions without relying on
> their SAML XML definitions.  Use of the registry by protocols other
> than SAML or OpenID Connect is encouraged.
> 
> While this is good in principle, and one presumes that each
> registration of sets of profiles from an existing protocol will be
> self-consistent, this text also encourages other (e.g., new)
> protocols to draw upon this registry without providing any
> guidance.  I'm concerned that it's probably possible to make a
> serious mess in a new protocol by using an LoA or two from
> multiple sets of registered LoAs without paying attention to
> whether the resulting collection of LoAs is consistent or coherent
> (or even sensible) for use in a single protocol.  This concern is
> reinforced by the guidance to expert reviewers in Section 4.1,
> which effectively conveys a desire to get all of the reasonable LoA
> profiles registered here, regardless of source or consistency with
> other registered LoA profiles.
> 
> I'd like to see some guidance to protocol designers and others for
> how to appropriately select multiple LoA profiles from this
> registry in a fashion that results in a consistent and (hopefully)
> usable collection.  For example, it may be a good idea to use (or
> start with) a set of related profiles already in use by an existing
> protocol in preference to mixing/matching individual profiles from
> multiple existing protocols.  At some level, this is common sense
> advice that the presence of profiles in this registry does not
> obviate the need to apply good design judgment, but that does
> deserve to be stated.

David,

The type of consistency you look for is extremely difficult to ascertain
and often rely on mapping the underlying policies. However I do see your
point. How about this:

"Protocol designers who want to reference this registry should be aware
that registered LoAs may depend on assumptions that do not carry over
to a new protocol."

> RFC 2119 keywords.  That's only a good idea in exceptional
> circumstances. I suggest removing section 1.1 and replacing upper
> case MUST/SHOULD/MAY with their lower case versions or reworded
> explanations of rationale.  Most of the uses of RFC 2119 keywords
> are not protocol requirements, but requirements on IANA,
> registrants, and users of the registry for which RFC 2119 keywords 
> Imperatives of the type defined in this memo must be used with
> care and sparingly.  In particular, they MUST only be used where it
> is actually required for interoperation or to limit behavior which
> has potential for causing harm (e.g., limiting retransmisssions)

ok

> 
> (2) Section 4
> 
> OLD The initial pool of expert and the review criteria are outlined
> below. NEW The review criteria are outlined below.
> 

good catch

> Nits/editorial comments:
> 
> Section 3
> 
> OLD The following ABNF productions represent reserved values and
> names NEW The reserved element defined by the following ABNF
> productions represents a set of reserved values and names

ok

> 
> Section 4
> 
> The registry is to be operated under the "Designated Expert
> Review" policy from RFC5226 [RFC5226] employing a pool of experts
> 
> Nope, the actual RFC5226 name of that well-known IANA policy is
> Expert Review (or Designated Expert), see section 4.1 of RFC5226.
> If that well-known IANA policy isn't what was intended, this is a
> serious open issue.

that IANA policy was indeed intended.

> 
> Top of p.7 The presense of an entry in the registy MUST NOT be
> taken to imply ^ r ---------------------------------------/
> 

Here I actually want normative language. It is quite important that the
registry not be over-interpreted.

> Section 7
> 
> OLD An implementor of MUST NOT treat the registry as a trust
> framework or NEW A protocol implementor MUST NOT treat the registry
> as a trust framework or
> 

I actually don't mean protocol implementor here. I mean consumer of the
registry.

> The minor issue about RFC 2119 keywords also applies to this text.

This is another case where I think I disagree!

> Thanks, --David

thank you!

	Cheers Leif

> ---------------------------------------------------- David L.
> Black, Distinguished Engineer EMC Corporation, 176 South St.,
> Hopkinton, MA  01748 +1 (508) 293-7953             FAX: +1 (508)
> 293-7786 [email-masked]        Mobile: +1 (978) 394-7754

---

From: <[email-masked]>
Date: Sun, 1 Apr 2012 23:58:09 -0400

Leif,

> that registered LoAs may depend on assumptions that do not carry over

Could you add "and that those assumptions may vary among the protocols for
which the LoAs were originally registered" ?

On RFC 2119 terms, I suggest that you talk to your AD (Sean).  The IESG
recently approved an IANA registry creation draft that I co-authored,
draft-ietf-storm-rddp-registries, and as part of that approval, removal of
RFC 2119 terms was requested.  See:

http://datatracker.ietf.org/doc/draft-ietf-storm-rddp-registries/history/

Thanks,
--David

> -----Original Message-----
> From: Leif Johansson [mailto:[email-masked]]
> Sent: Sunday, April 01, 2012 5:59 PM
> To: Black, David
> Cc: [email-masked]; [email-masked]; [email-masked]; [email-masked]; [email-masked]
> Subject: Re: Gen-ART review of draft-johansson-loa-registry-04
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
> 
> iEYEARECAAYFAk94z7cACgkQ8Jx8FtbMZnczSACgtOd/Ltv7PXDMYFkdbDHBeKdB
> n7UAoKNkSnBB/ZQZF96gwvKbTnXQq8Nt
> =lEQ5
> -----END PGP SIGNATURE-----

---

From: Leif Johansson <[email-masked]>
Date: Mon, 02 Apr 2012 09:06:57 +0200

On 04/02/2012 05:58 AM, [email-masked] wrote:
> Leif,
> 
>> The type of consistency you look for is extremely difficult to
>> ascertain and often rely on mapping the underlying policies.
>> However I do see your point. How about this:
>> 
>> "Protocol designers who want to reference this registry should be
>> aware that registered LoAs may depend on assumptions that do not
>> carry over to a new protocol."
> 
> Could you add "and that those assumptions may vary among the
> protocols for which the LoAs were originally registered" ?

yep

> 
> On RFC 2119 terms, I suggest that you talk to your AD (Sean).  The
> IESG recently approved an IANA registry creation draft that I
> co-authored, draft-ietf-storm-rddp-registries, and as part of that
> approval, removal of RFC 2119 terms was requested.  See:
> 
ok I'll do that. Thanks again!

> Thanks, --David
> 
>> -----Original Message----- From: Leif Johansson
>> [mailto:[email-masked]] Sent: Sunday, April 01, 2012 5:59 PM To:
>> Black, David Cc: [email-masked]; [email-masked];
>> [email-masked]; [email-masked]; [email-masked] Subject: Re:
>> Gen-ART review of draft-johansson-loa-registry-04
>> 
>>>> I am the assigned Gen-ART reviewer for this draft. For
>>>> background on Gen-ART, please see the FAQ at 
>>>> Document: draft-johansson-loa-registry-04 Reviewer: David L.
>>>> Black Review Date: April 1, 2012 IETF LC End Date: April 3,
>>>> 2012 IESG Telechat date: April 12, 2012
>>>> 
>>>> This draft establishes an IETF registry of SAML Level of
>>>> Assurance (LoA) profiles; it's short and clear, although it
>>>> does not contain any initial content for the registry -
>>>> presumably that will be supplied after the registry is
>>>> created via the expert review registration mechanism
>>>> established by this draft.
>>>> 
>>>> Summary: This draft is on the right track but has open
>>>> issues, described in the review.
>>>> 
>>>> Major issues: (1) My major open issue concerns the last
>>>> paragraph in the Introduction:
>>>> 
>>>> Authentication Context Profiles other protocols MAY use such
>>>> URIs to represent levels of assurance definitions without
>>>> relying on their SAML XML definitions.  Use of the registry
>>>> by protocols other than SAML or OpenID Connect is
>>>> encouraged.
>>>> 
>>>> registration of sets of profiles from an existing protocol
>>>> will be self-consistent, this text also encourages other
>>>> (e.g., new) protocols to draw upon this registry without
>>>> providing any guidance.  I'm concerned that it's probably
>>>> possible to make a serious mess in a new protocol by using an
>>>> LoA or two from multiple sets of registered LoAs without
>>>> paying attention to whether the resulting collection of LoAs
>>>> is consistent or coherent (or even sensible) for use in a
>>>> single protocol.  This concern is reinforced by the guidance
>>>> to expert reviewers in Section 4.1, which effectively conveys
>>>> a desire to get all of the reasonable LoA profiles registered
>>>> here, regardless of source or consistency with other
>>>> registered LoA profiles.
>>>> 
>>>> I'd like to see some guidance to protocol designers and
>>>> others for how to appropriately select multiple LoA profiles
>>>> from this registry in a fashion that results in a consistent
>>>> and (hopefully) usable collection.  For example, it may be a
>>>> good idea to use (or start with) a set of related profiles
>>>> already in use by an existing protocol in preference to
>>>> mixing/matching individual profiles from multiple existing
>>>> protocols.  At some level, this is common sense advice that
>>>> the presence of profiles in this registry does not obviate
> The type of consistency you look for is extremely difficult to
> ascertain and often rely on mapping the underlying policies.
> However I do see your point. How about this:
> 
> "Protocol designers who want to reference this registry should be
> aware that registered LoAs may depend on assumptions that do not
> carry over to a new protocol."
> 
>>>> 
>>>> Minor issues: (2)
>>>> 
>>>> (1) This draft is intended to be an informational RFC, but it
>>>> uses RFC 2119 keywords.  That's only a good idea in
>>>> exceptional circumstances. I suggest removing section 1.1 and
>>>> replacing upper case MUST/SHOULD/MAY with their lower case
>>>> versions or reworded explanations of rationale.  Most of the
>>>> uses of RFC 2119 keywords are not protocol requirements, but
>>>> requirements on IANA, registrants, and users of the registry
>>>> for which RFC 2119 keywords are not appropriate, e.g., see
>>>> RFC 2119 section 6:
>>>> 
>>>> Imperatives of the type defined in this memo must be used
>>>> with care and sparingly.  In particular, they MUST only be
>>>> used where it is actually required for interoperation or to
>>>> limit behavior which has potential for causing harm (e.g.,
>>>> limiting retransmisssions)
> 
> ok
> 
>>>> 
>>>> (2) Section 4
>>>> 
>>>> OLD The initial pool of expert and the review criteria are
>>>> outlined below. NEW The review criteria are outlined below.
>>>> 
>>>> OLD The following ABNF productions represent reserved values
>>>> and names NEW The reserved element defined by the following
>>>> ABNF productions represents a set of reserved values and
>>>> names
> 
> ok
> 
>>>> 
>>>> Section 4
>>>> 
>>>> Nope, the actual RFC5226 name of that well-known IANA policy
>>>> is Expert Review (or Designated Expert), see section 4.1 of
>>>> RFC5226. If that well-known IANA policy isn't what was
>>>> intended, this is a serious open issue.
> 
>>>> Top of p.7 The presense of an entry in the registy MUST NOT
>>>> be taken to imply ^ r
>>>> ---------------------------------------/
>>>> 
> 
> Here I actually want normative language. It is quite important that
> the registry not be over-interpreted.
> 
>>>> Section 7
>>>> 
>>>> framework or NEW A protocol implementor MUST NOT treat the
>>>> registry as a trust framework or
>>>> 
> 
> I actually don't mean protocol implementor here. I mean consumer of
> the registry.
> 
>>>> ---------------------------------------------------- David
>>>> L. Black, Distinguished Engineer EMC Corporation, 176 South
>>>> St., Hopkinton, MA  01748 +1 (508) 293-7953             FAX:
>>>> +1 (508) 293-7786 [email-masked]        Mobile: +1 (978)
>>>> 394-7754

---

From: <[email-masked]>
Date: Wed, 11 Apr 2012 17:49:23 -0400

The -05 version of this draft resolves all of the issues and comments raised
in the Gen-ART review of the -04 version.

Thanks,
--David

> From: Black, David
> Sent: Sunday, April 01, 2012 4:57 PM
> To: [email-masked]; [email-masked]; [email-masked]
> Cc: Black, David; Sean Turner; [email-masked]
> Subject: Gen-ART review of draft-johansson-loa-registry-04
>

---

From: Leif Johansson <[email-masked]>
Date: Wed, 11 Apr 2012 23:50:35 +0200

On 04/11/2012 11:49 PM, [email-masked] wrote:
> The -05 version of this draft resolves all of the issues and
> comments raised in the Gen-ART review of the -04 version.

that was quick! thanks

	Leif