Subject: [Gen-art] Re: Gen-ART review of draft-ietf-psamp-framework-12

From: Benoit Claise <[email-masked]>
Date: Wed, 16 Jan 2008 12:09:08 +0100

Hi Ben,

Thanks for your review.
See inline.
> I have been selected as the General Area Review Team (Gen-ART)
> reviewer for this draft (for background on Gen-ART, please see
> http://www.alvestrand.no/ietf/gen/art/gen-art-FAQ.html).
>
> Please resolve these comments along with any other Last Call comments
> you may receive.
>
>
> Document:  draft-ietf-psamp-framework-12
> Reviewer: Ben Campbell   
> Review Date:  10/4/2007
> IETF LC End Date: 10/5/2007
> IESG Telechat date: (if known)
>
> Summary:
>
> This document is almost ready for publication as in informational RFC, 
> but there is a potential open issue as well as a number of nits that 
> should be addressed prior to publication.
>
> Comments:
>
> Open Issue:
>
> Section 12:  Security Considerations
>
> The security consideration section seems too lightweight. It simply 
> calls out some security related requirements elsewhere in the 
> document, and offers no additional discussion.  It is not clear to me 
> that the referenced sections described the requirements from a 
> security perspective--it would be useful to describe in section 12 
> _why_ those requirements are important for security, and what 
> potential security issues or attacks they are intended to prevent.
>
> More importantly, PSAMP seems to me to have lots of privacy 
> implications, which the PSAMP charter appears to acknowledge with its 
> mention of privacy and anonymity issues. In my opinion these warrant 
> discussion in this section.
We're still discussing this issue amongst ourselves, and we will come 
back to you on this specific issue.
Let me address all the other issues in this email.
>
> nits:
>
> General:
>
> There are an unusually large number of authors listed in the Author 
> Addresses section.  Did all of these contribute substantial text to 
> the document?
This is up to editor Nick Duffield to decide if people contributed enough.
Note: I've been trying to contact Peram, who left Cisco. Peram never 
replied to me. As a consequence, I  think that we should move Peram in 
the acknowledgment section.
>
> There is a lot of language which appears to me to be normative, but 
> does not follow normal capitalization for normative statements. I 
> realize this is an informational document, but I gather the intent is 
> to define terms. The document often states that an element described 
> by a term "must" have certain features or meet certain requirements. 
> This feels like a place where normative language, with appropriate 
> capitalization makes sense. (This interacts with a specific comment on 
> section 2 below)
This is in line with the IPFIX architecture 
(http://tools.ietf.org/wg/ipfix/draft-ietf-ipfix-architecture/), which 
doesn't use the words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL 
NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" from 
RFC 2119.
>
> This document states a number of requirements that might be referenced 
> by future documents. 
Could you please give us an example.
> If there is any intent that other document be evaluated for compliance 
> with the requirements herein, it would be useful to break such 
> requirements out and number or otherwise label them so they can be 
> easily referenced in other documents.
>
> Section numbers are inconsistent in the use of a trailing period. This 
> makes searching for section headings difficult.
Agreed. Should be corrected with a new version of the draft.
>
> idnits returns the following warnings:
>
>  == Outdated reference: A later version (-08) exists of
>      draft-ietf-psamp-protocol-07
>
>   == Outdated reference: A later version (-07) exists of
>      draft-ietf-psamp-info-06
>
>   == Outdated reference: A later version (-26) exists of
>      draft-ietf-ipfix-protocol-24
Should be corrected with a new version of the draft.
>
>
> Abstract: Is it appropriate to give mail list information in an RFC? 
> This document will live forever, and I assume the mail list will 
> become obsolete some time before that.
The following lines should be removed:

      Comments on this document should be addressed to the PSAMP 
      Working Group mailing list: [email-masked] 
       
      To subscribe: [email-masked], in body: subscribe 
      Archive: https://ops.ietf.org/lists/psamp/ 

>
> After ToC but before section 1: There's a strange repeat of draft 
> boilerplate after the ToC but before the Intro. I assume it's a 
> formatting error.
Should be corrected with a new version of the draft.
>
> Section 2, first paragraph: "Definitions of terminology and the use of 
> the terms "must", "should" and "may" in this document are 
> informational only."
>
> I'm not sure what to make of this disclaimer. This document places a 
> number of requirements on devices it describes. Are those not 
> normative, at least in the sense that a device that does not meet 
> certain requirements should not be labeled a PSAMP device, etc?
See my answer above. We did this to be inline with IPFIX architecture draft.
>
>
> Bottom of page 9: ASCII art crosses the page boundary in a way likely 
> to confuse the reader.
Should be corrected with a new version of the draft.
>
> Section 4.4 Last Paragraph, last sentence: Comment about feasibility 
> and complexity of PSAMP operation feels like a non sequitur. Why is it 
> mentioned in a section about configuration?
"Feasibility and complexity of PSAMP operations is discussed in Section 
10." should be removed.
>
> Section 5.2: The section heading is orphaned from contents by a page 
> break. This can be confusing to the reader.
Should be corrected with a new version of the draft.
Anyway, this would be taken care of by the RFC-editor.
>
> Section 5.2 paragraphs 3 and 4: I suggest switching the order, since 
> the first refers to the second, and there appears to be no reason to 
> maintain the current order.
I would like to keep the order because they appear in the method 
numerical order (see the IPFIX-PROTO).
However, I agree the text should be changed.
OLD:

      * systematic count based sampling: similar to systematic time 
        based expect that selection is reckoned with respect to packet 
        count rather than time.  Packet selection is triggered 
        periodically by packet count, a number of successive packets 
        being selected subsequent to each trigger. 
    
      * systematic time based sampling: packet selection is triggered 
        at periodic instants separated by a time called the spacing.  
        All packets that arrive within a certain time of the trigger 
        (called the interval length) are selected. 

NEW

      * systematic count based sampling: packet selection is triggered 
        periodically by packet count, a number of successive packets 
        being selected subsequent to each trigger. 
    
      * systematic time based sampling: similar to systematic count 
        based expect that selection is reckoned with respect to time 
        rather than count.  Packet selection is triggered 
        at periodic instants separated by a time called the spacing.  
        All packets that arrive within a certain time of the trigger 
        (called the interval length) are selected. 

>
> Section 5.2 paragraph 5: n-out-of-N sampling: While this may be a term 
> of art of which I am not familiar, it's generally not a good idea to 
> distinguish two variable names by case alone.
I agree on the principle. However, n-out-of-N is the term used for years 
in the sampling world.
>
> Page 15, sixth paragraph: "When the PSAMP device offers property match 
> filtering..."
>
> This paragraph has interesting implications that a PSAMP device may 
> have other purposes other than metering, e.g. it may also be a router, 
> etc. While this may seem obvious to the writers, it may not to all 
> readers. I suggest explicitly mentioning that in the definition of 
> PSAMP device in section 3. Also, since the next few paragraphs talk 
> about how routers should expose information for PSAMP purposes, a 
> separate subsection talking specifically about routers acting as PSAMP 
> devices might be helpful.
I'm not sure I understand your comment, as the PSAMP device definition 
speaks about the router.

  3.7 PSAMP Device 
       
      A PSAMP Device is a device hosting at least an Observation Point, 
      a Selection Process and an Exporting Process.  Typically, 
      corresponding Observation Point(s), Selection Process(es) and 
      Exporting Process(es) are co-located at this device, for example 
      at a router. 

Here is a proposal anyway
OLD:

        When the PSAMP Device offers property match filtering, and, in 
        its usual capacity other than in performing PSAMP functions, 
        identifies or processes information from IP, transport or 
        encapsulation protocols, then the information should be made 
        available for filtering.  For example, when a PSAMP Device 
        routes based on destination IP address, that field should be 
        made available for filtering.  Conversely, a PSAMP Device that 
        does not route is not expected to be able to locate an IP 
        address within a packet, or make it available for Filtering, 
        although it may do so. 

NEW:

        When the PSAMP Device offers property match filtering, and, in 
        its usual capacity other than in performing PSAMP functions, 
        identifies or processes information from IP, transport or 
        encapsulation protocols, then the information should be made 
        available for filtering.  _For example, when the PSAMP Device 
        is a router that routes based on destination IP address_, that field should be 
        made available for filtering.  Conversely, a PSAMP Device that 
        does not route is not expected to be able to locate an IP 
        address within a packet, or make it available for Filtering, 
        although it may do so. 

Please let us know whether your comments are addressed.

Another change for the next version of the draft: some people changed affiliations.

Regards, Benoit.

_______________________________________________
Gen-art mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/gen-art

---

From: "DUFFIELD, NICHOLAS G (NICK)" <[email-masked]>
Date: Wed, 16 Jan 2008 10:24:06 -0500

Benoit: thanks. I can incorporate your proposed changes into the draft.

Ben: please let us know if you have any further feedback on Benoit's
proposed changes.

Nick

________________________________

From: Benoit Claise [mailto:[email-masked]] 
Sent: Wednesday, January 16, 2008 6:09 AM
To: Ben Campbell
Cc: General Area Review Team; [email-masked];
[email-masked]; [email-masked];
[email-masked]; DUFFIELD, NICHOLAS G (NICK); [email-masked];
[email-masked]; [email-masked]; Benoit Claise;
[email-masked]
Subject: Re: Gen-ART review of draft-ietf-psamp-framework-12

Hi Ben,

Thanks for your review. 
See inline.

Summary: 

Comments: 

Open Issue: 

The security consideration section seems too lightweight. It simply
calls out some security related requirements elsewhere in the document,
and offers no additional discussion.  It is not clear to me that the
referenced sections described the requirements from a security
perspective--it would be useful to describe in section 12 _why_ those
requirements are important for security, and what potential security
issues or attacks they are intended to prevent. 

nits: 

General: 

There are an unusually large number of authors listed in the Author
Addresses section.  Did all of these contribute substantial text to the
document? 

This is up to editor Nick Duffield to decide if people contributed
enough.
Note: I've been trying to contact Peram, who left Cisco. Peram never
replied to me. As a consequence, I  think that we should move Peram in
the acknowledgment section.

There is a lot of language which appears to me to be normative, but does
not follow normal capitalization for normative statements. I realize
this is an informational document, but I gather the intent is to define
terms. The document often states that an element described by a term
"must" have certain features or meet certain requirements. This feels
like a place where normative language, with appropriate capitalization
makes sense. (This interacts with a specific comment on section 2 below)

Abstract: Is it appropriate to give mail list information in an RFC?
This document will live forever, and I assume the mail list will become
obsolete some time before that. 

       

	After ToC but before section 1: There's a strange repeat of
draft boilerplate after the ToC but before the Intro. I assume it's a
formatting error. 

Section 2, first paragraph: "Definitions of terminology and the use of
the terms "must", "should" and "may" in this document are informational
only." 

I'm not sure what to make of this disclaimer. This document places a
number of requirements on devices it describes. Are those not normative,
at least in the sense that a device that does not meet certain
requirements should not be labeled a PSAMP device, etc? 

See my answer above. We did this to be inline with IPFIX architecture
draft.

Bottom of page 9: ASCII art crosses the page boundary in a way likely to
confuse the reader. 

Section 4.4 Last Paragraph, last sentence: Comment about feasibility and
complexity of PSAMP operation feels like a non sequitur. Why is it
mentioned in a section about configuration? 

Section 5.2 paragraphs 3 and 4: I suggest switching the order, since the
first refers to the second, and there appears to be no reason to
maintain the current order. 

    

NEW

	Section 5.2 paragraph 5: n-out-of-N sampling: While this may be
a term of art of which I am not familiar, it's generally not a good idea
to distinguish two variable names by case alone. 

This paragraph has interesting implications that a PSAMP device may have
other purposes other than metering, e.g. it may also be a router, etc.
While this may seem obvious to the writers, it may not to all readers. I
suggest explicitly mentioning that in the definition of PSAMP device in
section 3. Also, since the next few paragraphs talk about how routers
should expose information for PSAMP purposes, a separate subsection
talking specifically about routers acting as PSAMP devices might be
helpful. 

  3.7 PSAMP Device 

NEW:

        When the PSAMP Device offers property match filtering, and, in 
        its usual capacity other than in performing PSAMP functions, 
        identifies or processes information from IP, transport or 
        encapsulation protocols, then the information should be made 
        available for filtering.  For example, when the PSAMP Device 
        is a router that routes based on destination IP address, that
field should be 
        made available for filtering.  Conversely, a PSAMP Device that 
        does not route is not expected to be able to locate an IP 
        address within a packet, or make it available for Filtering, 
        although it may do so. 

Another change for the next version of the draft: some people changed
affiliations.

Regards, Benoit.

---

From: Ben Campbell <[email-masked]>
Date: Mon, 21 Jan 2008 15:29:25 -0600

Hi, sorry for the delayed response. Comments inline. (If I don't  
comment on something, please consider this an "okay").

On Jan 16, 2008, at 5:09 AM, Benoit Claise wrote:

>> This document is almost ready for publication as in informational  
>> RFC, but there is a potential open issue as well as a number of  
>> nits that should be addressed prior to publication.
>>
>> Comments:
>>
>> Open Issue:
>>
>> document, and offers no additional discussion.  It is not clear to  
>> me that the referenced sections described the requirements from a  
>> implications, which the PSAMP charter appears to acknowledge with  
>> its mention of privacy and anonymity issues. In my opinion these  
>> warrant discussion in this section.
> We're still discussing this issue amongst ourselves, and we will  
> come back to you on this specific issue.

[...]

>>
>> This document states a number of requirements that might be  
>> referenced by future documents.

Pretty much every bullet item in section 4 and its subsections.

[...]

>> Section 2, first paragraph: "Definitions of terminology and the use  
>> of the terms "must", "should" and "may" in this document are  
>> informational only."
>>
>> I'm not sure what to make of this disclaimer. This document places  
>> a number of requirements on devices it describes. Are those not  

Okay.

[...]

>> Section 5.2: The section heading is orphaned from contents by a  
>> page break. This can be confusing to the reader.

Never hurts to make their job easier :-)

>>
>> Section 5.2 paragraphs 3 and 4: I suggest switching the order,  
>> since the first refers to the second, and there appears to be no  
>> reason to maintain the current order.
That helps, thanks.

>>
>> Section 5.2 paragraph 5: n-out-of-N sampling: While this may be a  
>> term of art of which I am not familiar, it's generally not a good  
>> idea to distinguish two variable names by case alone.
> I agree on the principle. However, n-out-of-N is the term used for  
> years in the sampling world.

Okay-- I can live with it if it's a term of art.

>>
>> Page 15, sixth paragraph: "When the PSAMP device offers property  
>> match filtering..."
>>
>> have other purposes other than metering, e.g. it may also be a  
>> router, etc. While this may seem obvious to the writers, it may not  
>> to all readers. I suggest explicitly mentioning that in the  
>> definition of PSAMP device in section 3. Also, since the next few  
>> paragraphs talk about how routers should expose information for  
>> PSAMP purposes, a separate subsection talking specifically about  
>> routers acting as PSAMP devices might be helpful.
> I'm not sure I understand your comment, as the PSAMP device  
> definition speaks about the router.
>   3.7 PSAMP Device
>
>         is a router that routes based on destination IP address,  
> that field should be
That helps, thanks.

> Please let us know whether your comments are addressed.
I think so, with the exception of the known open item concerning  
privacy implications, and the clarification around which requirements  
I was referring to with my comment about numbering the requirements.

Thanks!

Ben.

---

From: Ben Campbell <[email-masked]>
Date: Mon, 21 Jan 2008 15:30:33 -0600

On Jan 16, 2008, at 9:24 AM, DUFFIELD, NICHOLAS G (NICK) wrote:

> Ben: please let us know if you have any further feedback on Benoit’s  
> proposed changes.

Only minor--please see my separate email to Benoit.

[...]

Thanks!

Ben.

---

From: Benoit Claise <[email-masked]>
Date: Mon, 28 Jan 2008 11:35:43 +0100

Hi Ben,

Thanks for your feedback. There are two open issues left in this email: 
open item concerning privacy implications and the clarification around 
requirements/ numbering. Let me address the second one here.

Your initial comment was:
This document states a number of requirements that might be referenced 
by future documents. If there is any intent that other document be 
evaluated for compliance with the requirements herein, it would be 
useful to break such requirements out and number or otherwise label them 
so they can be easily referenced in other documents.
> Pretty much every bullet item in section 4 and its subsections.
I understand what you mean now.
Our goal with the section 4 is the explanation of the requirements on 
which the architecture is based. In other words, a kind of introduction 
of what was important in terms of sampling/filtering/hashing. So we 
don't foresee that those requirements might be referenced by another 
document or protocol. As a consequence, we're not sure that numbering 
the requirements would help.

Regards, Benoit.

---

From: Ben Campbell <[email-masked]>
Date: Mon, 28 Jan 2008 05:50:27 -0600

On Jan 28, 2008, at 4:35 AM, Benoit Claise wrote:

> Hi Ben,
>
> Thanks for your feedback. There are two open issues left in this  
> email: open item concerning privacy implications and the  
> clarification around requirements/ numbering. Let me address the  
> second one here.
>
> referenced by future documents. If there is any intent that other  
> document be evaluated for compliance with the requirements herein,  
> it would be useful to break such requirements out and number or  
> otherwise label them so they can be easily referenced in other  
> documents.
>>
>>>>
> Our goal with the section 4 is the explanation of the requirements  
> on which the architecture is based. In other words, a kind of  
> introduction of what was important in terms of sampling/filtering/ 
> hashing. So we don't foresee that those requirements might be  
> referenced by another document or protocol. As a consequence, we're  
> not sure that numbering the requirements would help.

Okay, that's fine.

Thanks!

Ben.