Subject: Re: [CDNi] CDNI Logging : access to CDNI Logging File once it is moved from Subscription to Archive in the Logging Feed

From: Ben Niven-Jenkins <[email-masked]>
Date: Sun, 13 Oct 2013 08:27:57 +0100

Francois, Colleagues,

My opinion is that if logs are advertised in the subscription or archive feeds then they should be made available for download. I.e. we don't introduce new semantics that mean that if it is a CDN Logging feed then only entries in the subscription feed should be made available and the archive feed is "known" to probably point to dead links for log files.

Forcing a linkage between time covered by the subscription feed and tie and the retention time a uCDN would like a dCDN to provide is not a good idea IMO as it limits the flexibility we give to implementations for no advantage.

If a CDN wishes to remove old logs they should also remove the archive pages that reference those logs (or never generate/advertise them in the first place).

Ben

On 27 Sep 2013, at 14:15, Francois Le Faucheur (flefauch) wrote:

> Folks,
> 
> [as a co-author]
> 
> I'd like input on one of the very remaining open items in cdni-logging.
> 
> For memory, the CDNI Logging Feed allows a dCDN to advertise to the uCDN which CDNI Logging Files can be retrieved by the uCDN from the dCDN.
> It is structured as an Archived ATOM feed, meaning that a sliding window of currently available CDNI Logging files is advertised in a "Subscription" document, and then, over time, the older Logging Files are moved to an "Archive" document and new Logging Files appear in the Subscription document.
> 
> There is currently an editor's note asking a question about what should happen once a Logging File has been moved to an "archive":
> "
>  [Editor's note: if a given Logging file is moved away from
>   subscription document to an archive document, do we agree it may no
>   longer be accessible to uCDN?]
> "
> I personally agree that the Logging File need not be made available for download once it has moved away from the Subscripton document to the Archive document. My rationale is that:
> 	* this is precisely the point of the Subscription document i.e. to indicate which files are currently downloadable and the server-side is then responsible for ensuring it can serve these files (ie it has to retain them and be ready to serve them). 
> 	* it is not reasonable to expect that the server-side will make old Logging Files accessible forever
> 	* if the client-side wants to have a large window of time to pull Logging Files, then it needs to agree with the server-side and this period will be used as the retention period in the subscripton file. 
> 
> Please let us know if you disagree/agree with that, so we can address that in the next rev (this has not been addressed in -06 posted today).
> 
> For memory, I think the current text already implies the above:
> "
> The server-side implementation MUST respond to any valid pull request
>   by a client-side implementation for a CDNI Logging File published by
>   the server-side in the subscription document of the CDNI Logging
>   Feed. 
> "
> If we agree with the position above, we could also tweak the text to say that the server-side MUST respond _and_ actually provide the Logging File (ie responding 404 is not good).
> 
> Cheers
> 
> Francois
> 
> _______________________________________________
> CDNi mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/cdni

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Tue, 15 Oct 2013 08:52:37 +0000

Hi Ben,

> If a CDN wishes to remove old logs they should also remove the archive pages that reference those logs (or never generate/advertise them in the first place).

Please help me understand your proposal:

Say dCDN:
	* produced 24 Log files on 1 Oct , that were listed in an archive feed (archive-1-oct)
	* produced 24 Log files on 2 Oct, that were listed in an archive feed (archive-2-oct)
	* …
	* produced 24 Log files on 14 Oct, that were  listed in an archive feed (archive-14-oct)

Say today is 15 October and the dCDN has agreed with uCDN to retain log files for a period of one week.

So the Subscription feed has a prev-archive link to archive-14-oct, that has a prev-archive link to archive-13-oct, etc….. .
But what happens in archive-7-oct? does it need to be modified so that it no longer has a prev-archive link?
Or do you keep the prev-archive chaning forever and simply accept that some files advertised in an archive feed are no longer accessible (ie if uCDN tries pull a file advertised in archive-3-oct it will received a 404).?

I don't have a strong view on this topic but woudl like to document the chosen approach properly. 

Cheers

Francois

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 15 Oct 2013 11:23:19 +0100

Francois,

I believe the practice is to remove the prev-archive link in the (now) last archive feed page when previously last archive feed page is no longer available. 

Ben

> On 15 Oct 2013, at 09:52, "Francois Le Faucheur (flefauch)" <[email-masked]> wrote:
> 
> Hi Ben,
>

---

From: "Francois Le Faucheur (flefauch)" <[email-masked]>
Date: Tue, 15 Oct 2013 13:32:02 +0000

> I believe the practice is to remove the prev-archive link in the (now) last archive feed page when previously last archive feed page is no longer available. 

I just had a quick look at RFC5005.

It suggests that documents in an archive feed are permanent , which I understand to mean that they shoudl not be modified after having been sent, and that client will probably never pull them again. For example, it says:
	* "Archived feeds split the entries among multiple permanent documents"
	* " The requirement that archive documents be stable allows clients to safely assume that if they have retrieved one in the past, it will not meaningfully change in the future."

It also says that a publisher is not expected to serve all archives for-ever:
	* "Publishers are not required to make all archive documents available;
   they may refuse to serve (e.g., with HTTP status code 403 or 410) or
   be unable to serve (e.g., with HTTP status code 404) an archive
   document."

So, to me the behavior expected by RFC5005 would probably be to :
	1) just leave old archive feeds unmodified
	2) simply have the right to no longer serve old archive feeds when they are outside the time window agreed by uCDN and dCDN
	3) simply have the right to no longer server old CDNI logging files when they are outside the time window agreed by uCDN and dCDN

This seems in line with RFC5005 and its expectation that a client will not re-pull an old archive to see if it has been updated. It also seems very simple (ie nothing needs to ever get modified in the feed).

Could that work for you?

Francois

---

From: Ben Niven-Jenkins <[email-masked]>
Date: Tue, 15 Oct 2013 16:42:58 +0100

Francois,

> I just had a quick look at RFC5005.

<snip>

> This seems in line with RFC5005 and its expectation that a client will not re-pull an old archive to see if it has been updated. It also seems very simple (ie nothing needs to ever get modified in…

Yes, that works for me.

Ben