Subject: [MEDIACTRL] ISSUE 4 - IVR Package - <record> append

From: "McGlashan, Scott" <[email-masked]>
Date: Thu, 7 Oct 2010 18:10:50 +0000

Fourth (and final I think) issue for mailing list discussion following IESG review of the IVR package:

In 4.3.1.4:

   append:  indicates whether recorded data is appended or not to a
      recording location if a resource already exists.  A valid value is
      a boolean (see Section 4.6.1).  A value of true indicates that
      recorded data is appended to the existing resource at a recording
      location.  A value of false indicates that recorded data is to
      overwrite the existing resource.  The attribute is optional.  The
      default value is false.

How is append/overwrite mapped to underlying protocol being used?
In particular, I think this is underspecified in case of HTTP.

This is a good point and we don't say anything about how to map record upload to the underlying protocol. Since we mandated HTTP (and HTTPS), I'd prefer if we specify a mapping to HTTP. I'd propose something like:

When a recording location is specified using the HTTP or HTTPS protocol, the recording operation is mapped to the HTTP POST and PUT operations depending on whether the append attribute is true or false:

1. append = false. The recording data is uploaded to the specified location using HTTP PUT and replaces any data that location on the origin server. Success if the origin server returns 201 (Created); any other HTTP response is treated as an upload error.

2. append = true. The recording data is uploaded to the specified location using HTTP POST and is added to the resource - if any - at that location. Success if the origin server returns 204 (No Content) or 205 (Reset Content); any other HTTP response is treated as an upload error.

Let me know if you think this is along the right lines – any improvements (or other implementation experiences) welcome.  The other alternative is to delete the append attribute – especially if  no-one implements it .. If we do decide to delete the append attribute, I still think some text around the HTTP protocol mapping is useful.

Thanks

Scott

---

From: Stéphane Bastien <[email-masked]>
Date: Thu, 7 Oct 2010 11:25:33 -0700

Hi,

We use extensively the append attribute. We implemented this behavior:

1. append=false: Use HTTP PUT to overwrite the content of the URL after the recording is completed.

2. append=true: 
	a) Perform an HTTP GET on the URL
      b) Record new audio/video
      c) Append the new audio/video to the media file that was downloaded in step a)
      d) Perform an HTTP PUT to overwrite the old URL with the new media file.

Note that our implementation also matches the behavior expected for a media file stored on the local filesystem and accessed using the URI file://.

I do not agree with the proposal to enforce the usage of HTTP POST if append=true; because it makes the assumption that the remote HTTP server can append audio/video to an existing media file. The Media Server already knows how to handle audio tracks and video tracks, duplicating this functionality in the remote HTTP server seems counterproductive.

Stephane

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of McGlashan, Scott
Sent: 7 octobre 2010 14:11
To: [email-masked]
Cc: Melnikov Alexey; [email-masked]; [email-masked]
Subject: [MEDIACTRL] ISSUE 4 - IVR Package - <record> append

In 4.3.1.4:

Let me know if you think this is along the right lines - any improvements (or other implementation experiences) welcome.  The other alternative is to delete the append attribute - especially if  no-one implements it .. If we do decide to delete the append attribute, I still think some text around the HTTP protocol mapping is useful.

Thanks

Scott
_______________________________________________
MEDIACTRL mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/mediactrl
Supplemental Web Site:
http://www.standardstrack.com/ietf/mediactrl

---

From: Victor Paulsamy <[email-masked]>
Date: Fri, 8 Oct 2010 04:56:48 +0700

It should be possible to create new resource as well as append to an
existing one and hence, don't remove append option.

The text, for 1 and 2 below, satisfies my requirements.

--victor

On Fri, Oct 8, 2010 at 1:10 AM, McGlashan, Scott <[email-masked]>wrote:

> This is a good point and we don't say anything about how to map record
> upload to the underlying protocol. Since we mandated HTTP (and HTTPS), I'd
> prefer if we specify a mapping to HTTP. I'd propose something like:
>
> When a recording location is specified using the HTTP or HTTPS protocol,
> the recording operation is mapped to the HTTP POST and PUT operations
> depending on whether the append attribute is true or false:
>
> 1. append = false. The recording data is uploaded to the specified location
> using HTTP PUT and replaces any data that location on the origin server.
> Success if the origin server returns 201 (Created); any other HTTP response
> is treated as an upload error.
>
> 2. append = true. The recording data is uploaded to the specified location
> using HTTP POST and is added to the resource - if any - at that location.
> Success if the origin server returns 204 (No Content) or 205 (Reset
> Content); any other HTTP response is treated as an upload error.
>
> Let me know if you think this is along the right lines – any improvements
> (or other implementation experiences) welcome.  The other alternative is to
> delete the append attribute – especially if  no-one implements it .. If we
> do decide to delete the append attribute, I still think some text around the
> HTTP protocol mapping is useful.
>
> Thanks
>
> Scott

---

From: Lorenzo Miniero <[email-masked]>
Date: Fri, 8 Oct 2010 13:11:35 +0200

I agree with Stephane, appending to a media file is not like appending
text and so a "brutal" POST might result in a corrupted file.

That said, I think append is a very useful functionality and would like
to keep it: why not clarify that the mechanism used for appending is
implementation specific? I doubt the same recorded media will be used
by different MS implementations at the same time.

L.

On Thu, 7 Oct 2010 11:25:33 -0700

-- 
Lorenzo Miniero
Meetecho s.r.l.
http://www.meetecho.com

---

From: Victor Paulsamy <[email-masked]>
Date: Sat, 9 Oct 2010 07:09:39 +0700

2010/10/8 Lorenzo Miniero <[email-masked]>

> text and so a "brutal" POST might result in a corrupted file.
Are you anticipating corruption or have you encountered it?

In my testing, during the last 8-months or so, append works like a charm --
even under load; I'm yet to run into recorded media corruption. We download
files only for playback. Recorded media are transferred to remote (HTTP or
NFS) storage, in chunks of about 40-ms audio, without storing it onboard
(append is heavily used.)

However, I'm not against leaving the append mechanism to implementers.

--victor

> > Note that our implementation also matches the behavior expected for a
> media file stored on the local filesystem and accessed using the URI
> file://.
> >
> > I do not agree with the proposal to enforce the usage of HTTP POST if
> append=true; because it makes the assumption that the remote HTTP server can
> append audio/video to an existing media file. The Media Server already knows
> how to handle audio tracks and video tracks, duplicating this functionality
> in the remote HTTP server seems counterproductive.
> >
> > Stephane
> >
> >
> > Cc: Melnikov Alexey; [email-masked];
> [email-masked]
> > 1. append = false. The recording data is uploaded to the specified
> location using HTTP PUT and replaces any data that location on the origin
> server. Success if the origin server returns 201 (Created); any other HTTP
> response is treated as an upload error.
> >
> >
> > 2. append = true. The recording data is uploaded to the specified
> location using HTTP POST and is added to the resource - if any - at that
> location. Success if the origin server returns 204 (No Content) or 205
> (Reset Content); any other HTTP response is treated as an upload error.
> >
> > Let me know if you think this is along the right lines - any improvements
> delete the append attribute - especially if  no-one implements it .. If we

---

From: Lorenzo Miniero <[email-masked]>
Date: Sat, 9 Oct 2010 10:44:09 +0200

Hi Victor, see inline.

On Sat, 9 Oct 2010 07:09:39 +0700

> Are you anticipating corruption or have you encountered it?

Just anticipating actually: I guess it really depends on the format and encoding used for the media to append to, if both audio and video are involved, if source and target are of the same type and so on.

-- 
Lorenzo Miniero <[email-masked]>

---

From: Stéphane Bastien <[email-masked]>
Date: Sat, 9 Oct 2010 04:58:50 -0700

Hi,

Just appending data to the end of media files work for some audio file formats, but fails for common file formats such as MOV, 3GP, and MP4. (Even appending audio to a WAV file fails unless you modify the WAV header - which means that the HTTP server handling the POST must know how to do this.) And as Lorenzo mentions in another e-mail, how can the HTTP POST deal with issues such as the first part of a recording is in one codec, while the remainder of the recording is using some other codec. This is just one more failure mode when using plain HTTP POST for appending.

I strongly recommend that we leave the appending mechanism as an implementation-specific issue in the spec.

Stéphane

Le 2010-10-08 à 20:09, Victor Paulsamy a écrit :

2010/10/8 Lorenzo Miniero <[email-masked]<mailto:[email-masked]>>
I agree with Stephane, appending to a media file is not like appending
text and so a "brutal" POST might result in a corrupted file.

In my testing, during the last 8-months or so, append works like a charm -- even under load; I'm yet to run into recorded media corruption. We download files only for playback. Recorded media are transferred to remote (HTTP or NFS) storage, in chunks of about 40-ms audio, without storing it onboard (append is heavily used.)

---

From: Eric Burger <[email-masked]>
Date: Tue, 12 Oct 2010 08:39:31 -0400

Will we run into interoperability problems if this is implementation-dependent? My guess is if it is up to the media server, the media server already knows the current file format and codec, so should also be able to know if it can just add bytes to the end of the current file or if it has to suck down the whole thing, repackage, and send it back.

---

From: Steve Liao <[email-masked]>
Date: Tue, 12 Oct 2010 14:08:43 -0700

On the contrary, I believe we can run into media file viability problems if we mandate mapping append=true to HTTP POST.  As Stephane and Lorenzo stated, adding to a media file oftentimes requires detailed knowledge of the media format and oftentimes cannot be accomplished by simply adding bytes to the end of an existing file.

While it's quite conceivable for a Media Server to perform the container format conversions and/or transcodings as necessary to append media, this capability is not necessarily available on an HTTP server.

Therefore, how the Media Server provides append=true should be left up to implementation, so long as it satisfies the requirement of append=true resulting in a media file that properly incorporates the new media.

Regards,
Steve

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Eric Burger [[email-masked]]
Sent: Tuesday, October 12, 2010 5:39 AM
To: [email-masked]
Subject: Re: [MEDIACTRL] ISSUE 4 - IVR Package - <record> append

---

From: Eric Burger <[email-masked]>
Date: Wed, 13 Oct 2010 06:26:24 -0500

I think this means we agree: since the Media Server already knows the structure of the container format and codec, and the Media Server knows what the very semantics of "append" means for the container and codec, that we should leave it up to the Media Server to decide. So long as on the other side it is just stock, RFC 2616 HTTP, I think we should be fine.

---

From: Steve Liao <[email-masked]>
Date: Wed, 13 Oct 2010 11:08:38 -0700

Agreed.

Thanks,
Steve