Subject: Re: [CDNi] I-D Action: draft-bertrand-cdni-logging-02.txt

From: <[email-masked]>
Date: Mon, 5 Nov 2012 15:22:05 +0000

David,

Thanks for your review of our draft. 

1) We want the WG to discuss the choice of a Logging transport protocol for non real-time information. Personally, I do not yet have a clear preference for a given transport protocol. We definitely need to identify the pro / cons of the candidate protocols and I would like to invite the WG contributors to share their technical arguments on the mailing list. 

2) Please see 1)   :-) 
  
3) Thanks for sharing these arguments. 

  To clarify the scope of the open technical questions:
        - I think there is consensus on the fact that the logging information required by CDNs in the scope of CDNI in quite specific. So, we need to define a   Logging format.
	  - There is no consensus for the moment on the choice of a Logging transport protocol for non real-time information. 
	    - The candidate protocols are: SNMPv3, Syslog, SSH File Transport, HTTPS. Did I forget any relevant protocol?
          - Your point is that SNMPv3 and syslog (and any other candidate protocol) must not be excluded without documenting technical reasons. I definitely agree.

Note that we want to have a clear separation between the logging application layer (the format of the information) and the logging transport layer. So, the WG may decide to support more than on logging transport protocol.

Best regards,

Gilles

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de ietfdbh
Envoyé : lundi 22 octobre 2012 23:17
À : [email-masked]
Cc : David Harrington
Objet : Re: [CDNi] I-D Action: draft-bertrand-cdni-logging-02.txt

Hi,

I took a quick look at this document.
I have a few comments.

1) While appendix C is yet to be written, the placeholders seem to already poo-poo the candidates that are existing IETF standards, without documenting why they are not viable choices.

2) Appendix C2 references RFC6707, which states some conclusions that have no documentation as to how the conclusions were reached.
What are the scalability requirements for the CDNI logging interface? 
RFC6707 states that there are "scalability concerns", but doesn't specify the requirements or the specific concerns. 
There may be valid scalability concerns, but this deserves better documentation than mere handwaving.
Which scalability requirements are not possible to meet with SNMPv3?

3) RFC6707 states SNMP does not support guaranteed delivery of traps. 
I am not sure how this conclusion was reached.
SNMPv3 has Informs (acknowledged traps), which support application-level acknowledgement of receipt. 
An SNMPv3 agent could continue to periodically send the Inform until it receives acknowledgement of receipt.
This would seem to mitigate the concern that use of SNMP "could result in log records being lost". 
Note that SNMPv1 has been declared Historic, so shouldn't even be considered a candidate protocol for new usage. 
Only version 3 of SNMP is recommended.

That said, I would think that syslog [RFC5424] might be a better choice for CDNI logging.
It uses secure, reliable transport over TLS (RFC5425).
Signed syslog messages [RFC5848] can be used by receivers to verify they have received all the messages sent to them.
If messages are out-of-order or missing, the receiver can determine which records are missing and request they be resent.
In addition, the signing can be applied to stored logs to verify they have not been modified from the original message stream.

If you are considering the RECOMMENDED versions of these candidate protocols, they might better meet the requirements.
But the main point is that the consideration process should be documented more clearly than this document (or RFC6707) does.

David Harrington
[email-masked]
+1-603-828-1401
-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of [email-masked]
Sent: Monday, October 22, 2012 2:59 PM
To: [email-masked]
Subject: I-D Action: draft-bertrand-cdni-logging-02.txt

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title           : CDNI Logging Interface
	Author(s)       : Gilles Bertrand
                          Stephan Emile
                          Roy Peterkofsky
                          Francois Le Faucheur
                          Pawel Grochocki
	Filename        : draft-bertrand-cdni-logging-02.txt
	Pages           : 43
	Date            : 2012-10-22

Abstract:
   This memo specifies the Logging interface between a downstream CDN
   (dCDN) and an upstream CDN (uCDN) that are interconnected as per the
   CDN Interconnection (CDNI) framework.  First, it describes a
   reference model for CDNI logging.  Then, it specifies the actual
   protocol for CDNI logging information exchange covering the
   information elements as well as the transport of those.

The IETF datatracker status page for this draft is:
https://datatracker.ietf.org/doc/draft-bertrand-cdni-logging

There's also a htmlized version available at:
http://tools.ietf.org/html/draft-bertrand-cdni-logging-02

A diff from the previous version is available at:
http://www.ietf.org/rfcdiff?url2=draft-bertrand-cdni-logging-02

Internet-Drafts are also available by anonymous FTP at:
ftp://ftp.ietf.org/internet-drafts/

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html or ftp://ftp.ietf.org/ietf/1shadow-sites.txt

_______________________________________________
CDNi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/cdni

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Gene Golovinsky <[email-masked]>
Date: Mon, 5 Nov 2012 07:34:18 -0800

Hi.

Some questions re usage of Syslog RFC 5424 were asked at the previous
meeting. Then, were also raised on the list. I did not see a technical
conversation about it. (May be I missed it). The same applies to SNMP
Informs.
I would too suggest that Syslog may be a better choice since it allows for
easier extension and longer massages.

The only argument I have heard against these two so far is that CDNi logs
may be too long.

Cheers.
--Gene

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of
[email-masked]
Sent: Monday, November 05, 2012 07:22 AM
To: ietfdbh; [email-masked]
Subject: Re: [CDNi] I-D Action: draft-bertrand-cdni-logging-02.txt

David,

1) We want the WG to discuss the choice of a Logging transport protocol
for non real-time information. Personally, I do not yet have a clear
preference for a given transport protocol. We definitely need to identify
the pro / cons of the candidate protocols and I would like to invite the
WG contributors to share their technical arguments on the mailing list.

2) Please see 1)   :-)

  To clarify the scope of the open technical questions:
        - I think there is consensus on the fact that the logging
information required by CDNs in the scope of CDNI in quite specific. So,
we need to define a   Logging format.
	  - There is no consensus for the moment on the choice of a
Logging transport protocol for non real-time information.
	    - The candidate protocols are: SNMPv3, Syslog, SSH File
Transport, HTTPS. Did I forget any relevant protocol?
          - Your point is that SNMPv3 and syslog (and any other candidate
protocol) must not be excluded without documenting technical reasons. I
definitely agree.

Note that we want to have a clear separation between the logging
application layer (the format of the information) and the logging
transport layer. So, the WG may decide to support more than on logging
transport protocol.

Best regards,

Gilles

-----Message d'origine-----
De : [email-masked] [mailto:[email-masked]] De la part de
ietfdbh Envoyé : lundi 22 octobre 2012 23:17 À : [email-masked] Cc : David
Harrington Objet : Re: [CDNi] I-D Action:
draft-bertrand-cdni-logging-02.txt

Hi,

1) While appendix C is yet to be written, the placeholders seem to already
poo-poo the candidates that are existing IETF standards, without
documenting why they are not viable choices.

2) Appendix C2 references RFC6707, which states some conclusions that have
no documentation as to how the conclusions were reached.
What are the scalability requirements for the CDNI logging interface?
RFC6707 states that there are "scalability concerns", but doesn't specify
the requirements or the specific concerns.
There may be valid scalability concerns, but this deserves better
documentation than mere handwaving.
Which scalability requirements are not possible to meet with SNMPv3?

3) RFC6707 states SNMP does not support guaranteed delivery of traps.
I am not sure how this conclusion was reached.
SNMPv3 has Informs (acknowledged traps), which support application-level
acknowledgement of receipt.
An SNMPv3 agent could continue to periodically send the Inform until it
receives acknowledgement of receipt.
This would seem to mitigate the concern that use of SNMP "could result in
log records being lost".
Note that SNMPv1 has been declared Historic, so shouldn't even be
considered a candidate protocol for new usage.
Only version 3 of SNMP is recommended.

That said, I would think that syslog [RFC5424] might be a better choice
for CDNI logging.
It uses secure, reliable transport over TLS (RFC5425).
Signed syslog messages [RFC5848] can be used by receivers to verify they
have received all the messages sent to them.
If messages are out-of-order or missing, the receiver can determine which
records are missing and request they be resent.
In addition, the signing can be applied to stored logs to verify they have
not been modified from the original message stream.

If you are considering the RECOMMENDED versions of these candidate
protocols, they might better meet the requirements.
But the main point is that the consideration process should be documented
more clearly than this document (or RFC6707) does.

David Harrington
[email-masked]
+1-603-828-1401
-----Original Message-----

A New Internet-Draft is available from the on-line Internet-Drafts
directories.

Abstract:

_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/i-d-announce
Internet-Draft directories: http://www.ietf.org/shadow.html or
ftp://ftp.ietf.org/ietf/1shadow-sites.txt

__________________________________________________________________________
_______________________________________________

Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
exploites ou copies sans autorisation. Si vous avez recu ce message par
erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les
pieces jointes. Les messages electroniques etant susceptibles
d'alteration, France Telecom - Orange decline toute responsabilite si ce
message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged
information that may be protected by law; they should not be distributed,
used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for
messages that have been modified, changed or falsified.
Thank you.

---

From: Kevin J Ma <[email-masked]>
Date: Mon, 5 Nov 2012 12:14:41 -0500

Hi All,

  I had a couple questions about the logging config and the use of the control
  interface rather than the metadata interface for distributing the config.

  - Though the config is static, it may differ for different pieces of content?
    The metadata interface already includes provisions for apply properties to
    specific sets of content based on URI path; logging could be a property.

  - Though the filtering could occur higher up than the surrogates, is that
    going to be efficient?  The surrogates would have to log all fields for
    the upstream filtering function to work.  The extra metadata seems small
    in comparison to the extra (unnecessary) logging data sent to the filter?

    Furthermore, if a CP wants an additional field (e.g., X-vendor-blah header)
    that would need to be conveyed to the surrogates to be logged?

thanx.

-- Kevin J. Ma

> Sent: Monday, October 22, 2012 3:06 PM
> To: [email-masked]
> Subject: [CDNi] New version of the CDNI Logging draft posted
> 
> Folks,
> 
> We have submitted a new version of the CDNI logging draft:
> URL:             http://www.ietf.org/internet-drafts/draft-bertrand-cdni-
> logging-02.txt
> Status:          http://datatracker.ietf.org/doc/draft-bertrand-cdni-
> logging
> Htmlized:        http://tools.ietf.org/html/draft-bertrand-cdni-logging-02
> Diff:            http://www.ietf.org/rfcdiff?url2=draft-bertrand-cdni-
> logging-02
> 
> Best regards,
> 
> Gilles
> 
> De : [email-masked] [mailto:[email-masked]]
> De la part de [email-masked]
> Envoyé : lundi 22 octobre 2012 20:59
> À : [email-masked]
> Objet : I-D Action: draft-bertrand-cdni-logging-02.txt
> 
> 
> confidentielles ou privilegiees et ne doivent donc
> pas etre diffuses, exploites ou copies sans autorisation. Si vous avez
> recu ce message par erreur, veuillez le signaler
> a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
> electroniques etant susceptibles d'alteration,
> information that may be protected by law;

---

From: "ietfdbh" <[email-masked]>
Date: Mon, 5 Nov 2012 13:36:31 -0500

Hi,

How long are CDNI log messages expected to be?
What are the requirements to be met in terms of confidentiality, integrity
checking, etc.?

RFC5424 syslog deliberately does not hard-code the length.
For interoperability, senders MUST support at least 2048, receivers SHOULD
support at least 8192 bytes.
For usages that need larger message sizes, such as HIPPA audit logs, you can
use larger message sizes.
However, each syslog sender/receiver/relay/application in the path to
deliver that data would need to support these larger sizes to avoid
truncation.

SNMP is designed to support traps as alerts, and to support trap-directed
polling.
Traps/Informs are typically small messages to alert the NMS on a timely
basis to some condition; small alerts typically better meet operational
alerting requirements.
You typically do not send lots of data in a trap/inform.
If there is a lot of information to be passed, a trap can be used to tell
the NMS something is available, and the NMS can then poll for the actual
data (or use a different protocol to get the data).
So, depending on what you plan to log, and the importance of timely
reporting, traps/informs may or may not meet operational requirements.

David Harrington
[email-masked]
+1-603-828-1401
-----Original Message-----
From: Gene Golovinsky [mailto:[email-masked]] 
Sent: Monday, November 05, 2012 10:34 AM
To: [email-masked]; ietfdbh; [email-masked]
Subject: RE: [CDNi] I-D Action: draft-bertrand-cdni-logging-02.txt

Hi.

Cheers.
--Gene

-----Original Message-----

David,

1) We want the WG to discuss the choice of a Logging transport protocol for
non real-time information. Personally, I do not yet have a clear preference
for a given transport protocol. We definitely need to identify the pro /
cons of the candidate protocols and I would like to invite the WG
contributors to share their technical arguments on the mailing list.

2) Please see 1)   :-)

	  - There is no consensus for the moment on the choice of a Logging
transport protocol for non real-time information.

Note that we want to have a clear separation between the logging application
layer (the format of the information) and the logging transport layer. So,
the WG may decide to support more than on logging transport protocol.

Best regards,

Gilles

-----Message d'origine-----

Hi,

1) While appendix C is yet to be written, the placeholders seem to already
poo-poo the candidates that are existing IETF standards, without documenting
why they are not viable choices.

Note that SNMPv1 has been declared Historic, so shouldn't even be considered
a candidate protocol for new usage.

That said, I would think that syslog [RFC5424] might be a better choice for
CDNI logging.

David Harrington
[email-masked]
+1-603-828-1401
-----Original Message-----

Abstract:

Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc pas etre diffuses,
exploites ou copies sans autorisation. Si vous avez recu ce message par
erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les
pieces jointes. Les messages electroniques etant susceptibles d'alteration,
France Telecom - Orange decline toute responsabilite si ce message a ete
altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged
information that may be protected by law; they should not be distributed,
used or copied without authorisation.
If you have received this email in error, please notify the sender and
delete this message and its attachments.
As emails may be altered, France Telecom - Orange is not liable for messages
that have been modified, changed or falsified.
Thank you.

---

From: Kevin J Ma <[email-masked]>
Date: Tue, 6 Nov 2012 18:38:08 -0500

Hi All,

  A couple of additional thoughts on metadata vs control for logging config:

  - While I can see an argument for using control to convey uCDN to dCDN
    logging information, because it is not content specific, it is not
    clear to me why we would need to do that logging at all.  Both the
    uCDN and the dCDN know what was transfered; the sharing of logs does
    not seem that meaningful.  Unlike when the dCDN talks to the client,
    and needs to tell the uCDN about it.

  - If we consider the control interface to be triggers, it is not clear
    that the semantics of triggers lends itself to pushing persistent
    information.  Whereas, with metadata, the logging information for a
    given piece of content would be automatically accessed on demand
    whenever the content is accessed by an end user, rather than having
    to be proactively pushed by the uCDN and maintained by the dCDN.

  - Finally, even if we consider that the optimal logging config for a
    given surrogate is the aggregate of the different logging configs
    for the different content it may server, combined with the internal
    CDN logging needs, this optimization can still be performed by the
    local CDN.  Just as with content acquisition, I would not expect all
    the surrogates use the uCDN metadata directly; I would expect some type
    of hierarchical organization of servers through which the surrogates
    acquire the content.  The metadata interface conveys inter-CDN data;
    intra-CDN per-surrogate optimization can still be performed by the
    local CDN, outside the scope of CDNI.

thanx.

--  Kevin J. Ma

> From: Kevin J Ma
> Sent: Monday, November 05, 2012 12:15 PM
> To: [email-masked]'; [email-masked]
> Subject: RE: New version of the CDNI Logging draft posted
> 
> Hi All,
>

---

From: Roy Peterkofsky <[email-masked]>
Date: Tue, 06 Nov 2012 17:48:19 -0800

Kevin,

The rational for using the control interface was that we deemed it 
unlikely that, in practice, uCDNs and dCDNs would engage in the practice 
of specifying the elements to be logged and transmitted independently 
for each request or piece of content.  What seems much more likely is 
that uCDNs and dCDNs will agree in advance about which elements will be 
transmitted and then adopt that standard for all traffic transacted 
between the two parties.  To control the logged elements on a 
per-content or per-request basis would:

1) Add a much higher level of complexity to the task of capturing and 
transmitting log data on the part of the logging CDN.  Most edge server 
streaming software, for example, does not have logic to capture 
different data elements for different pieces of content.
2) Imply that the CDNs that provide reports to CSPs or to internal 
departments are going to provide different reports to those 
constituencies about different content, which would also add a great 
deal of complexity and would be a use this that we have not seen.  CDNs 
typically offer the same portfolio of reports to all of their CSP 
customers, for example.
3) Add unnecessary volume to the metadata traffic for each content element.

While we can see some cases where logging and reporting may differ for 
different types of content (e.g., HAS vs. HTTP small object delivery), 
this is not likely to vary by specific content element within a type of 
content.

For these reasons, we did not think that is was most appropriate to put 
these transactions in the content-specific metadata interface.

It is certainly conceivable that, someday, some CDNs might want the 
capability to specify logging elements differently for each content 
element.  But we do not know of any CDN platform providers that are even 
considering this capability on their near-term or strategic roadmaps.

Roy

On 11/6/2012 3:38 PM, Kevin J Ma wrote:
> -----
> No virus found in this message.
> Checked by AVG - www.avg.com
> Version: 2013.0.2742 / Virus Database: 2617/5877 - Release Date: 11/06/12
>
>

-- 
Roy Peterkofsky
Vice President, Product Management
Skytide -- the leader in Digital Media Performance Management
www.skytide.com
(510) 250-4284

Read our new white paper: The 4 Keys to Telco CDN Success 
<http://www.slideshare.net/skytide/the-4-keys-to-telco-cdn-success>

---

From: Kevin J Ma <[email-masked]>
Date: Wed, 7 Nov 2012 00:46:16 -0500

Hi Roy,

> The rational for using the control interface was that we deemed it unlikely that, in practice, uCDNs and dCDNs would engage in the practice of specifying the elements to be logged and transmitted independently for each request or piece of content.

I'm not sure what you mean by "specifying the elements to be logged and
transmitted independently for each request"?  I don't see why anyone would
want to change the logging config on each request.  The metadata interface
is not really for that purpose in any case.  As for specifying logging config
for "each piece of content", I would agree that doesnt make sense, but that
does not mean all content should have the same logs.  The metadata interface
is hierarchical and supports either global config or config for some subset
based on URI wildcard match.  It should be flexible and scalable.

> 1) Add a much higher level of complexity

One might argue that it allows for a much more efficient method of logging
where surrogates would not need to log unnecessary data?

> 2) Imply that the CDNs that provide reports to CSPs or to internal departments are going to provide different reports to those constituencies about different content

We may or may not.  I'm not sure why we would just rule it out?
Isn't the purpose of filtering, in the draft, to generate different reports?

> 3) Add unnecessary volume to the metadata traffic for each content element.

I do not see it being that much metadata traffic.  I do not see the logging
config being different for every piece of content, but I would also not want
to constrain all logging to be the same for every piece of content.  Metadata
is specified by URI path match (with wildcards).  I would expect every mpegts
segment for a given CP to have the same config, not a config per segment?
That should only be one piece of metadata for all those mpegts segments?
Having segment specific metadata would not really scale in general.

thanx.

--  Kevin J. Ma

From: Roy Peterkofsky [mailto:[email-masked]]
Sent: Tuesday, November 06, 2012 8:48 PM
To: Kevin J Ma
Cc: [email-masked]; [email-masked]
Subject: Re: [CDNi] New version of the CDNI Logging draft posted

Kevin,

The rational for using the control interface was that we deemed it unlikely that, in practice, uCDNs and dCDNs would engage in the practice of specifying the elements to be logged and transmitted independently for each request or piece of content.  What seems much more likely is that uCDNs and dCDNs will agree in advance about which elements will be transmitted and then adopt that standard for all traffic transacted between the two parties.  To control the logged elements on a per-content or per-request basis would:

1) Add a much higher level of complexity to the task of capturing and transmitting log data on the part of the logging CDN.  Most edge server streaming software, for example, does not have logic to capture different data elements for different pieces of content.
2) Imply that the CDNs that provide reports to CSPs or to internal departments are going to provide different reports to those constituencies about different content, which would also add a great deal of complexity and would be a use this that we have not seen.  CDNs typically offer the same portfolio of reports to all of their CSP customers, for example.
3) Add unnecessary volume to the metadata traffic for each content element.

While we can see some cases where logging and reporting may differ for different types of content (e.g., HAS vs. HTTP small object delivery), this is not likely to vary by specific content element within a type of content.

For these reasons, we did not think that is was most appropriate to put these transactions in the content-specific metadata interface.

It is certainly conceivable that, someday, some CDNs might want the capability to specify logging elements differently for each content element.  But we do not know of any CDN platform providers that are even considering this capability on their near-term or strategic roadmaps.

Roy

Hi All,

thanx.

--  Kevin J. Ma

-----Original Message-----

From: Kevin J Ma

To: [email-masked]<mailto:[email-masked]>'; [email-masked]<mailto:[email-masked]>

Hi All,

  I had a couple questions about the logging config and the use of the

control

  interface rather than the metadata interface for distributing the

config.

  - Though the config is static, it may differ for different pieces of

content?

    The metadata interface already includes provisions for apply

properties to

    specific sets of content based on URI path; logging could be a

property.

  - Though the filtering could occur higher up than the surrogates, is

that

    going to be efficient?  The surrogates would have to log all fields

for

    the upstream filtering function to work.  The extra metadata seems

small

    in comparison to the extra (unnecessary) logging data sent to the

filter?

    Furthermore, if a CP wants an additional field (e.g., X-vendor-blah

header)

thanx.

-- Kevin J. Ma

-----Original Message-----

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]] On Behalf Of

[email-masked]<mailto:[email-masked]>

To: [email-masked]<mailto:[email-masked]>

Folks,

URL:             http://www.ietf.org/internet-drafts/draft-bertrand-

cdni-

logging-02.txt

logging

Htmlized:        http://tools.ietf.org/html/draft-bertrand-cdni-logging-

02

logging-02

Best regards,

Gilles

-----Message d'origine-----

De : [email-masked]<mailto:[email-masked]> [mailto:i-d-announce-

[email-masked]<mailto:[email-masked]>]

De la part de [email-masked]<mailto:[email-masked]>

À : [email-masked]<mailto:[email-masked]>

directories.

                          Stephan Emile

                          Roy Peterkofsky

                          Francois Le Faucheur

                          Pawel Grochocki

    Pages           : 43

    Date            : 2012-10-22

Abstract:

[email-masked]<mailto:[email-masked]>

Thank you.

CDNi mailing list

[email-masked]<mailto:[email-masked]>

CDNi mailing list

[email-masked]<mailto:[email-masked]>

-----

Checked by AVG - www.avg.com<http://www.avg.com>

--
Roy Peterkofsky
Vice President, Product Management
Skytide -- the leader in Digital Media Performance Management
www.skytide.com<http://www.skytide.com>
(510) 250-4284

Read our new white paper: The 4 Keys to Telco CDN Success<http://www.slideshare.net/skytide/the-4-keys-to-telco-cdn-success>

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Wed, 7 Nov 2012 05:49:07 +0000

> The rational for using the control interface was that we deemed it unlikely that, in practice, uCDNs and dCDNs would engage in the practice of specifying the elements to be logged and transmitted i…

I can't see uCDNs and dCDNs engaging in the practice of specifying elements to log for each request or piece of content but I can see folks wanting to log different sets of fields for different hostnames.

I think there is also an architectural element to this as well. IMO CDNI Metadata should be used to convey all the 'configuration' related to content delivery. Which fields to log is part of that configuration IMO.

>  What seems much more likely is that uCDNs and dCDNs will agree in advance about which elements will be transmitted and then adopt that standard for all traffic transacted between the two parties.  To control the logged elements on a per-content or per-request basis would:
> 

Except logs aren't just about reporting. There is likely to be a 'core' set of log fields a uCDN requires for generating its reports but there are other fields that get logged purely for debugging for example. Some of those fields are generic across hostnames some are specific to a particular service running on a subset of hostnames.

> 3) Add unnecessary volume to the metadata traffic for each content element.

I'm not convinced the additional volume is really all that great, especially if you consider logging per hostname as opposed to logging per content element.

One approach is certainly for the dCDN to log for each hostname the superset of all log fields required for all hostnames delegated by that uCDN, i.e. what you are proposing.

If different hostnames only require a subset of the superset of log fields the dCDN is logging then you've just traded a small amount of volume on the CDNI Metadata interface (a list of log fields per hostname) for a potentially huge amount of log data (all the log fields a particular CSP doesn't care about but gets anyway because a different CSP using the same uCDN does care about them).

It could be that that is a tradeoff we're happy to make, but I think it needs more discussion first.

Ben

>>> To: '
>>> [email-masked]'; [email-masked]
>>> 
>>>> From: 
>>>> [email-masked] [mailto:[email-masked]
>>>> ] On Behalf Of
>>>> 
>>>> To: 
>>>> [email-masked]
>>>> URL:             
>>>> http://www.ietf.org/internet-drafts/draft-bertrand
>>>> -
>>>> 
>>> cdni-
>>> 
>>>> logging-02.txt
>>>> Status:          
>>>> http://datatracker.ietf.org/doc/draft-bertrand-cdni
>>>> -
>>>> logging
>>>> Htmlized:        
>>>> http://tools.ietf.org/html/draft-bertrand-cdni-logging
>>>> -
>>>> 
>>> 02
>>> 
>>>> De : 
>>>> [email-masked] [mailto:i-d-announce
>>>> -
>>>> 
>>> [email-masked]
>>> ]
>>> 
>>>> À : 
>>>> [email-masked]
>>>> Internet-Draft directories: 
>>>> http://www.ietf.org/shadow.html
>>>>  or
>>>>

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Wed, 7 Nov 2012 16:21:13 +0000

Good discussion.
The next rev of the logging draft will start documenting the requirements for long exchange, specifically in terms of scalability and reliability.

Quick question on an assumption below:
> 
> But as an example, let's assume a cache is delivering Live MS Smoothstreaming and the clients are requesting the manifest every 10 seconds then a CDN will be generating 1100 log lines a second for each Gbps of traffic it is delivering.

So there are some assumption here on some average number of simultaneous viewers of that Live stream, right?
Just to understand this figure, can you expand a bit on how this has been derived, like is this based on observation in some production CDN over a number of Live channels?

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Wed, 21 Nov 2012 12:12:12 +0000

> Just to understand this figure, can you expand a bit on how this has been derived, like is this based on observation in some production CDN over a number of Live channels? 

Sorry, I got my maths wrong. I was assuming each client was receiving an average of 1 Mbps with 2 second chunks and a manifest request every 10 seconds  so 1000 clients per Gbps of traffic requesting a chunk each every 2 seconds and a manifest request each every 10 seconds, so 600 requests/s on average.

Ben

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Wed, 21 Nov 2012 13:59:04 +0000

> Sorry, I got my maths wrong. I was assuming each client was receiving an average of 1 Mbps with 2 second chunks and a manifest request every 10 seconds  so 1000 clients per Gbps of traffic requesti…

Makes sense now.

---

From: Roy Peterkofsky <[email-masked]>
Date: Wed, 21 Nov 2012 11:59:16 -0800

If you have 1000 clients concurrently, you can expect one video AND one 
audio fragment every two seconds per client = 1000 total reqs/second.  
If you add in your assumed 1 manifest request per 10 seconds per client 
( x 1000 clients), that is another 100 reqs/second.  Net effect, I think 
your original 1100 lines/second was correct (under the stated assumptions).

On 11/21/2012 4:12 AM, Ben Niven-Jenkins wrote:
> Version: 2013.0.2793 / Virus Database: 2629/5909 - Release Date: 11/21/12
>
>