Subject: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

From: "Law, Will" <[email-masked]>
Date: Mon, 17 Aug 2020 23:30:01 +0000

Hello

I am requesting a clarification on the expected server response status code when a client makes an open-ended range request against a media segment when playing back LL-HLS.  Consider the following media playlist snippet:

#EXTINF:4.000,
v1_1-7727.m4s
#EXT-X-PROGRAM-DATE-TIME:2020-07-28T17:37:48.771Z
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="64267@0",INDEPENDENT=YES
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="63033@64267"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="57810@127300"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60558@185110"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="68575@245668"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60880@314243"
#EXT-X-PRELOAD-HINT:TYPE=PART,URI="v1_1-7728.m4s",BYTERANGE-START=375123

To start-up, the client would issue 6 GET range-requests , starting at the independent part The server would respond with a 206 response for each and a content-range response header indicating the range being returned.

The next request would be for the PRELOAD-HINT part. This would be a GET request with a “range: 375123-“. The client is indicating it wants to receive this object starting at offset 375123 and continuing to the end of the segment.

How should the origin (or proxy server) respond to this PRELOAD request? Three possible options

  1.  It holds back any response until the end of segment, returning at that time a 206 response with a content-range of 375123 – T/T+1 (where represents total size of segment). This would ruin the low latency behavior for the client.
  2.  It starts an immediate response, signaling 206 with content-range: 375123 - */*. This is actually forbidden by the RFC’s, which indicate that the last-byte-pos cannot hold a value of “*”.
  3.  It starts an immediate response, signaling a 200 response and if serving H1 to a proxy server which then serves H2 to the client, a "Transfer-Encoding: chunked" response header.

Option [3] looks to be the most correct however it raises the question of  whether a 200 response (chunked-transfer or not) implies to the client that is has received the complete object i.e starting at offset 0 instead of offset 375123. As a CDN, we need to build a behavior that is robustly supported by all HTTP clients and not just a particular class of application. The proxy-server cannot tell if is serving a LL-HLS client or some other client, therefore we need a consistent behavior when it is asked for an open-ended range request against an object of unknown size.

So the questions are:

  1.  Is a 200 status-code expected in the response to the PRELOAD request?
  2.  If so, are there other clients or applications on the internet that would break if we did so for all open-ended range requests against objects of unknown size (outside of the application space of LL-HLS)?

Cheers
Will

---

From: Mark Nottingham <[email-masked]>
Date: Tue, 18 Aug 2020 09:36:54 +1000

Hi Will,

Not sure it's exactly applicable to your use case, but does this help?
  https://tools.ietf.org/html/rfc8673#section-2.2

Cheers,

> On 18 Aug 2020, at 9:30 am, Law, Will <wilaw=[email-masked]> wrote:
> 
> Hello
>  
> 	• It holds back any response until the end of segment, returning at that time a 206 response with a content-range of 375123 – T/T+1 (where represents total size of segment). This would ruin the low latency behavior for the client.
> 	• It starts an immediate response, signaling 206 with content-range: 375123 - */*. This is actually forbidden by the RFC’s, which indicate that the last-byte-pos cannot hold a value of “*”. 
> 	• It starts an immediate response, signaling a 200 response and if serving H1 to a proxy server which then serves H2 to the client, a "Transfer-Encoding: chunked" response header. 
>  
> 	• Is a 200 status-code expected in the response to the PRELOAD request?
> 	• If so, are there other clients or applications on the internet that would break if we did so for all open-ended range requests against objects of unknown size (outside of the application space of LL-HLS)?
>  
> Cheers
> Will
>  
>  
>  
> -- 
> Hls-interest mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/hls-interest

--
Mark Nottingham   https://www.mnot.net/

---

From: Lucas Pardue <[email-masked]>
Date: Tue, 18 Aug 2020 00:38:09 +0100

Would RFC 8673 [1] fit the requirement here?

> To accommodate byte-range requests for content that has data appended
over time, this document defines semantics that allow an HTTP client and a
server to perform byte-range GET and HEAD requests that start at an
arbitrary byte offset within the representation and end at an indeterminate
offset.

[1] https://www.rfc-editor.org/rfc/rfc8673.html

On Tue, 18 Aug 2020, 00:30 Law, Will, <wilaw=[email-masked]>
wrote:

> Hello
>
>
>
> I am requesting a clarification on the expected server response status
> code when a client makes an open-ended range request against a media
> segment when playing back LL-HLS.  Consider the following media playlist
> snippet:
>
>
>
> #EXTINF:4.000,
>
> v1_1-7727.m4s
>
> To start-up, the client would issue 6 GET range-requests , starting at the
> independent part The server would respond with a 206 response for each and
> a content-range response header indicating the range being returned.
>
>
>
> The next request would be for the PRELOAD-HINT part. This would be a GET
> request with a “range: 375123-“. The client is indicating it wants to
> receive this object starting at offset 375123 and continuing to the end of
> the segment.
>
>
>
>    1. It holds back any response until the end of segment, returning at
>    that time a 206 response with a content-range of 375123 – T/T+1 (where
>    represents total size of segment). This would ruin the low latency behavior
>    for the client.
>    2. It starts an immediate response, signaling 206 with content-range:
>    375123 - */*. This is actually forbidden by the RFC’s, which indicate that
>    the last-byte-pos cannot hold a value of “*”.
>    3. It starts an immediate response, signaling a 200 response and if
>    serving H1 to a proxy server which then serves H2 to the client, a
>    "Transfer-Encoding: chunked" response header.
>
>
>
> Option [3] looks to be the most correct however it raises the question of
>  whether a 200 response (chunked-transfer or not) implies to the client
> that is has received the complete object i.e starting at offset 0 instead
> of offset 375123. As a CDN, we need to build a behavior that is robustly
> supported by all HTTP clients and not just a particular class of
> application. The proxy-server cannot tell if is serving a LL-HLS client or
> some other client, therefore we need a consistent behavior when it is asked
> for an open-ended range request against an object of unknown size.
>
>
>
> So the questions are:
>
>    2. If so, are there other clients or applications on the internet that
>    would break if we did so for *all open-ended range requests against
>    objects of unknown size* (outside of the application space of LL-HLS)?
>
>
>
> Cheers
>
> Will
>
>
>
>
>
>
> --

---

From: Pieter-Jan Speelmans <[email-masked]>
Date: Tue, 18 Aug 2020 06:52:22 +0200

Hi all

Related to that:

> a content-range response header indicating the range being returned.

An optimisation we have considered from a client side would be to identify
those 6 range requests would actually fold together in one larger range
request (and potentially the request for the preload hint as well). Folding
the first six requests together should be OK according to normal HTTP
flows.

What would be the expected behaviour however if the preload-hint request
also gets collapsed into this or simply the asset v1_1-7728.m4s would be
requested without a range request at all?

Best regards,
Pieter-Jan

*Pieter-Jan Speelmans*CTO

*THEO Technologies NV*
*w *| www.theoplayer.com
Philipssite 5, Bus 1. 3001 Leuven, Belgium
<https://d2oj23ymxjbl3l.cloudfront.net/>
Leuven (Belgium) - New York (US) - San Francisco (US) - Singapore (SG) -
Barcelona (ES)
BTW: BE 0847.829.290, RPR: Leuven

-- 
*Legal Notice *
You can find the latest terms and policies of THEO 
Technologies under www.theoplayer.com/terms 
<http://www.theoplayer.com/terms> including our "GENERAL TERMS AND 
CONDITIONS". In the absence of a signed agreement between you and THEO 
Technologies, by replying to this email these terms apply to the relevant 
transaction between us.   
The contents of this email message and any 
attachments are intended solely for the addressee(s) and contain 
CONFIDENTIAL and/or privileged information and are legally protected from 
disclosure. If you are not the intended recipient of this message, please 
immediately alert the sender by reply email and then delete this message 
including any attachments and you are hereby notified that any use, 
dissemination, copying, or storage of this message or its attachments is 
strictly prohibited. 
Full security of emails over the internet cannot be 
ensured. Despite our efforts it is your responsibility to provide for your 
protection.
Global HQ: THEO Technologies NV, Philipssite 5 bus 1, 3001 
Heverlee, Belgium - BE 0847.829.290 - CEO: Steven Tielemans

---

From: "Law, Will" <[email-masked]>
Date: Tue, 18 Aug 2020 18:06:31 +0000

@Mark<mailto:[email-masked]> @Lucas – thank you both for your quick responses, both pointing at RFC8673 https://www.rfc-editor.org/rfc/rfc8673.html. This does address the exact problem. The solution is outlined below for those who want a primer. It revolves around the client signaling a magic number last-byte-pos which is “much larger than the last-byte-pos returned in response to an open-ended byte-range HEAD request, as described above, and much larger than the expected maximum size of the representation.” . This leads to the obvious problem of what threshold value should an origin or CDN use? A stateless edge server has no idea what is “the expected maximum size of the representation”.

All CDNs should use the same threshold value, for consistency of client responses. Additionally, all LL-HLS origins should also apply the same threshold, for the same reason. The RFC makes a recommendation to use 2^^53-1 (9007199254740991) as the threshold. It is Javascript safe and in fact corresponds to Number.MAX_SAFE_INTEGER. Akamai is implementing this as we speak, and our current intent is to use 9007199254740991. However some flexibility to align with the status quo and we are willing to align with other CDNs for industry consistency in support.

@Mark – what threshold value does Fastly use?

@Lucas – what threshold value does Cloudflare use?

@Other CDN implementers on this list – what value do you use?

Equally, all LL-HLS clients should use the same threshold value. For those building LL-HLS clients that support byte range addressing, any objections to using 9007199254740991?

----------------------------------------------------------------

A client can indicate support for handling indeterminate-length byte-range responses by providing a very large value for the last-byte-pos in the byte-range request. For example, a client can perform a byte-range GET request of the form:

GET /resource HTTP/1.1
Host: example.com
Range: bytes=1230000-999999999999

where the last-byte-pos in the request is much larger than the last- byte-pos returned in response to an open-ended byte-range HEAD request, as described above, and much larger than the expected maximum size of the representation. See Section 6 for range value considerations.

In response, a server may indicate that it is supplying a continuously aggregating/live response by supplying the client request's last-byte-pos in the Content-Range response header field.

For example:

GET /resource HTTP/1.1
Host: example.com
Range: bytes=1230000-999999999999

returns

HTTP/1.1 206 Partial Content
Content-Range: bytes 1230000-999999999999/*

from the server to indicate that the response will start at byte
1230000 and continue indefinitely to include all aggregated content,
as it becomes available.

Cheers
Will

From: Pieter-Jan Speelmans <[email-masked]>
Date: Monday, August 17, 2020 at 9:53 PM
To: "Law, Will" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Subject: Re: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

Hi all

Related to that:

To start-up, the client would issue 6 GET range-requests , starting at the independent part The server would respond with a 206 response for each and a content-range response header indicating the range being returned.
An optimisation we have considered from a client side would be to identify those 6 range requests would actually fold together in one larger range request (and potentially the request for the preload hint as well). Folding the first six requests together should be OK according to normal HTTP flows.

What would be the expected behaviour however if the preload-hint request also gets collapsed into this or simply the asset v1_1-7728.m4s would be requested without a range request at all?

Best regards,
Pieter-Jan

Pieter-Jan Speelmans
CTO

THEO Technologies NV
w | www.theoplayer..com<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theoplayer.com_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=zRH5_eqUYPpnZnmTuXadhCfqsPz4Kf1T4_9-KmoNRpk&e=>
Philipssite 5, Bus 1. 3001 Leuven, Belgium
[Image removed by sender.]<https://urldefense.proofpoint.com/v2/url?u=https-3A__d2oj23ymxjbl3l.cloudfront.net_&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=Q6XrRZNHy3KpCquvR83tEmqOc-klGILP6gPYJegRx7s&e=>
Leuven (Belgium) - New York (US) - San Francisco (US) - Singapore (SG) - Barcelona (ES)
BTW: BE 0847.829.290, RPR: Leuven

On Tue, Aug 18, 2020 at 1:30 AM Law, Will <wilaw=[email-masked]<mailto:[email-masked]>> wrote:
Hello

#EXTINF:4..000,
v1_1-7727..m4s

So the questions are:

Cheers
Will

--
Hls-interest mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hls-interest<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=lAcAl0Wr7tRdhayCf3crO76tH1jgoJAQ8ant2PDXpCA&e=>

Legal Notice
You can find the latest terms and policies of THEO Technologies under www.theoplayer.com/terms<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.theoplayer.com_terms&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=3V-L-7irWJaIKqd70_fh4S30y0kfh-uHlFMaZnMQJfo&s=raivU9TwEr7KoP4i_LeGjwPIRM5Tn_6GWXUoLm5wiN4&e=> including our "GENERAL TERMS AND CONDITIONS". In the absence of a signed agreement between you and THEO Technologies, by replying to this email these terms apply to the relevant transaction between us.
The contents of this email message and any attachments are intended solely for the addressee(s) and contain CONFIDENTIAL and/or privileged information and are legally protected from disclosure. If you are not the intended recipient of this message, please immediately alert the sender by reply email and then delete this message including any attachments and you are hereby notified that any use, dissemination, copying, or storage of this message or its attachments is strictly prohibited.
Full security of emails over the internet cannot be ensured. Despite our efforts it is your responsibility to provide for your protection.
Global HQ: THEO Technologies NV, Philipssite 5 bus 1, 3001 Heverlee, Belgium - BE 0847.829.290 - CEO: Steven Tielemans

---

From: Roger Pantos <[email-masked]>
Date: Wed, 19 Aug 2020 13:17:26 -0700

> On Aug 17, 2020, at 4:30 PM, Law, Will <wilaw=[email-masked]> wrote:
> 
> Hello
>  
> It holds back any response until the end of segment, returning at that time a 206 response with a content-range of 375123 – T/T+1 (where represents total size of segment). This would ruin the low latency behavior for the client.
> It starts an immediate response, signaling 206 with content-range: 375123 - */*. This is actually forbidden by the RFC’s, which indicate that the last-byte-pos cannot hold a value of “*”. 
> It starts an immediate response, signaling a 200 response and if serving H1 to a proxy server which then serves H2 to the client, a "Transfer-Encoding: chunked" response header. 
>  

Note that when using H2 (or H3) that there is a fourth option, which is to return 206 and not supply any Content-Length header at all. This (I believe) became the expected behavior when HTTP/2 removed support for chunked transfer encoding: https://httpwg.org/specs/rfc7540.html#HttpSequence

Roger Pantos
Apple Inc. 

> Is a 200 status-code expected in the response to the PRELOAD request?
> If so, are there other clients or applications on the internet that would break if we did so for all open-ended range requests against objects of unknown size (outside of the application space of LL-HLS)?
>  
> Cheers
> Will
>  
>  
>  
> -- 
> [email-masked] <mailto:[email-masked]>
> https://www.ietf.org/mailman/listinfo/hls-interest <https://www.ietf.org/mailman/listinfo/hls-interest>

---

From: "Law, Will" <[email-masked]>
Date: Thu, 20 Aug 2020 00:11:41 +0000

@Roger – re “Note that when using H2 (or H3) that there is a fourth option, which is to return 206 and not supply any Content-Length header at all.”
This is a good point however it still leaves open the question of what content-range header should be returned. Per the H2 spec (snippet below), the requirements of HTTP/1.1 Range Requests are carried forward under HTTP/2. These requirements RFC7233 state that the “the server generating the 206 response MUST generate a Content-Range header field,” and also that the byte-range must contain a last-byte-pos value and that a value of * can be used for an unknown total length of the object but it cannot be used for last-byte-pos. Since the last-byte-pos is not known in this case, we have the same problem with H2 that we have with H1.

The solution proposed by https://tools.ietf.org/html/rfc8673 would seem to work for H2 as well as it would for H1. Per this solution, the client should never make an open ended range request if it is expecting an aggregated response from a fixed offset. It should instead send a request with a very large number (9007199254740991 has been proposed in this thread) as the last-byte-pos in the range request. This would signal the server (or origin) to begin a response that starts at the requested offset and aggregates over time until the object is completely transferred.

https://tools.ietf.org/html/rfc7540#section-8

8. HTTP Message Exchanges

HTTP/2 is intended to be as compatible as possible with current uses
of HTTP. This means that, from the application perspective, the
features of the protocol are largely unchanged. To achieve this, all
request and response semantics are preserved, although the syntax of
conveying those semantics has changed.

Thus, the specification and requirements of HTTP/1.1 Semantics and
Content [RFC7231], Conditional Requests [RFC7232], Range Requests
[RFC7233], Caching [RFC7234], and Authentication [RFC7235] are
applicable to HTTP/2.

-Will

From: Roger Pantos <rpantos=[email-masked]>
Date: Wednesday, August 19, 2020 at 1:17 PM
To: "[email-masked]" <[email-masked]>
Subject: Re: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

On Aug 17, 2020, at 4:30 PM, Law, Will <wilaw=[email-masked]<mailto:wilaw=[email-masked]>> wrote:

Hello

#EXTINF:4.000,
v1_1-7727.m4s

Note that when using H2 (or H3) that there is a fourth option, which is to return 206 and not supply any Content-Length header at all. This (I believe) became the expected behavior when HTTP/2 removed support for chunked transfer encoding: https://httpwg.org/specs/rfc7540.html#HttpSequence<https://urldefense.proofpoint.com/v2/url?u=https-3A__httpwg.org_specs_rfc7540.html-23HttpSequence&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=QsYGd9vU3GGNCIKQjBOnvGSnXVh73OH0Kyi1KcPWpDw&e=>

Roger Pantos
Apple Inc.

So the questions are:

Cheers
Will

--
Hls-interest mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hls-interest<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=Yv0CtXmSurk_NBFYprpu4mN6giQ1RFARMSK8zxyR6Rg&e=>

---

From: Roger Pantos <[email-masked]>
Date: Tue, 25 Aug 2020 12:38:13 -0700

Okay, I think that approach would work well enough. We’ll take a closer look at it once iOS 14 et al are in the can. Assuming it works out we can put a reference to that part of RFC 8673 into the EXT-X-PRELOAD-HINT section of the HLS spec. 

We can also update our clients to conform to that behavior, although it might take a while to land that in an iOS/macOS release.

Roger.

> On Aug 19, 2020, at 5:11 PM, Law, Will <wilaw=[email-masked]> wrote:
> 
> The solution proposed by https://tools.ietf.org/html/rfc8673 <https://tools.ietf.org/html/rfc8673> would seem to work for H2 as well as it would for H1. Per this solution, the client should never make an open ended range request if it is expecting an aggregated response from a fixed offset. It should instead send a request with a very large number (9007199254740991 has been proposed in this thread) as the last-byte-pos in the range request. This would signal the server (or origin) to begin a response that starts at the requested offset and aggregates over time until the object is completely transferred.
> 
>  
> https://tools.ietf.org/html/rfc7540#section-8 <https://tools.ietf.org/html/rfc7540#section-8>
>> On Aug 17, 2020, at 4:30 PM, Law, Will <wilaw=[email-masked] <mailto:wilaw=[email-masked]>> wrote:
>>  
>> Hello
>>  
> Note that when using H2 (or H3) that there is a fourth option, which is to return 206 and not supply any Content-Length header at all. This (I believe) became the expected behavior when HTTP/2 removed support for chunked transfer encoding: https://httpwg.org/specs/rfc7540.html#HttpSequence <https://urldefense.proofpoint.com/v2/url?u=https-3A__httpwg.org_specs_rfc7540.html-23HttpSequence&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=QsYGd9vU3GGNCIKQjBOnvGSnXVh73OH0Kyi1KcPWpDw&e=>
>  
>  
> Roger Pantos
> Apple Inc. 
> 
> 
>> https://www.ietf.org/mailman/listinfo/hls-interest <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=Yv0CtXmSurk_NBFYprpu4mN6giQ1RFARMSK8zxyR6Rg&e=>
> 
> 
> --

---

From: "Law, Will" <[email-masked]>
Date: Wed, 26 Aug 2020 15:44:21 +0000

@Roger – that’s good news. A clear directive within the spec will help achieve consistent player behavior as well as consistent CDN and origin response.

One note on the applicability of RFC 8673 would be that it seems to apply for any request for LL-HLS content that is open-ended, not only for PRELOAD-HINT requests. The interesting thing about range-based addressing with LL-HLS is that the player on average only need make one request per media type per segment duration. This is true in nearly all start-up and switch conditions too, with the sole exception being when the PRELOAD-HINT represents the first part in a new segment. The following playlist snippets illustrate this point. Can you confirm if this is the expected client behavior?

Case #1:
#EXTINF:4.000,
v1_1-7727.m4s

A player seeing this playlist at startup would need to make a single request of the form

   GET / v1_1-7728.m4s HTTP/2
   Host: example.com
   Range: bytes=0-9007199254740991

The origin would respond by bursting the bytes it has (up to 375123) and then releasing the remainder as each part boundary becomes available. This would give it the independent part it needs to start, plus all the segments up to and including the HINTed part.

  HTTP/2 206 Partial Content
   Content-Range: bytes 0-9007199254740991/*

Case #2:

#EXTINF:4.000,
v1_1-7727.m4s
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="68575@245668",INDEPENDENT=YES

A player seeing this playlist at startup would make a single request of the form

   GET / v1_1-7728.m4s HTTP/2
   Host: example.com
   Range: bytes=245668-9007199254740991

Origin would respond by bursting the bytes from 245668 to 375123 and then releasing the remainder as each part boundary becomes available.

  HTTP/2 206 Partial Content
   Content-Range: bytes 245668-9007199254740991/*

Case #3:

#EXTINF:4.000,
v1_1-7727.m4s
#EXT-X-PROGRAM-DATE-TIME:2020-07-28T17:37:48.771Z
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="64267@0",INDEPENDENT=YES
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="63033@64267"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="57810@127300"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60558@185110"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="68575@245668",INDEPENDENT=YES
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="60880@314243"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="62485@375123"
#EXT-X-PART:DURATION=0.500,URI="v1_1-7728.m4s",BYTERANGE="61325@437608"
#EXTINF:4.000,
v1_1-7728.m4s
#EXT-X-PRELOAD-HINT:TYPE=PART,URI="v1_1-7729.m4s",BYTERANGE-START=0

This player would make a first request for

   GET / v1_1-7728.m4s HTTP/2
   Host: example.com
   Range: bytes=245668-498933

Note that this request is NOT using the RFC8673 convention for last-byte-pos, because the last-byte-pos is known. The origin would respond by adding in a content-length response header, signaling the total bytes in the content-range header and bursting all these bytes since all the content is fully available.

  HTTP/2 206 Partial Content
  Content-Length: 253265
  Content-Range: bytes 245668-498933/498934

The player would then make a second open ended range request for

   GET / v1_1-7729.m4s HTTP/2
   Host: example.com
   Range: bytes=0-9007199254740991

Origin would respond by releasing the content as each part becomes available.

  HTTP/2 206 Partial Content
   Content-Range: bytes 0-9007199254740991/*

This is the steady state condition of the player from this point on. It keeps making a series of open-ended range requests against each new segment starting at an offset of 0. This continues until it needs to switch, encounters a discontinuity or playback pauses.  During this time it continues to update the media playlist at a period of the part duration, in order to maintain its knowledge of the internal structure of each segment and also to discover discontinuities.

Cheers
Will

Date: Tuesday, August 25, 2020 at 12:38 PM

Roger.

On Aug 19, 2020, at 5:11 PM, Law, Will <wilaw=[email-masked]<mailto:wilaw=[email-masked]>> wrote:

The solution proposed by https://tools.ietf.org/html/rfc8673<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc8673&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=oMR4zfPXrho88f5S0AzpDC5z3D0bwKPKsOm2kymBHBk&e=> would seem to work for H2 as well as it would for H1. Per this solution, the client should never make an open ended range request if it is expecting an aggregated response from a fixed offset. It should instead send a request with a very large number (9007199254740991 has been proposed in this thread) as the last-byte-pos in the range request. This would signal the server (or origin) to begin a response that starts at the requested offset and aggregates over time until the object is completely transferred.

https://tools.ietf.org/html/rfc7540#section-8<https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7540-23section-2D8&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=BzKvOAFjcWwd1oNZnRb_NH1b50kZYcJ245JTW83T53g&e=>

-Will

From: Roger Pantos <rpantos=[email-masked]<mailto:rpantos=[email-masked]>>
Date: Wednesday, August 19, 2020 at 1:17 PM
To: "[email-masked]<mailto:[email-masked]>" <[email-masked]<mailto:[email-masked]>>
Subject: Re: [Hls-interest] LL-HLS: status-code expected in the response to the PRELOAD request

Hello

#EXTINF:4.000,
v1_1-7727.m4s

Roger Pantos
Apple Inc.

So the questions are:

Cheers
Will

--
Hls-interest mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/hls-interest<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=VZVvigVlbA5cXwltSzsJSJ8ZrOEueLMD1jVYQO51fRk&e=>

---

From: Pieter-Jan Speelmans <[email-masked]>
Date: Wed, 26 Aug 2020 17:50:33 +0200

Hi Will

I would indeed expect this behavior to be the case if the client is
configured to collapse part requests when the part hold off is configured
to three part duration.

P

On Wed, 26 Aug 2020, 17:44 Law, Will, <wilaw=[email-masked]>
wrote:

> @Roger – that’s good news. A clear directive within the spec will help
> achieve consistent player behavior as well as consistent CDN and origin
> response.
>
>
>
> One note on the applicability of RFC 8673 would be that it seems to apply
> for *any request for LL-HLS content that is open-ended,* not only for
> PRELOAD-HINT requests. The interesting thing about range-based addressing
> with LL-HLS is that the player on average only need make one request per
> media type per segment duration. This is true in nearly all start-up and
> switch conditions too, with the sole exception being when the PRELOAD-HINT
> represents the first part in a new segment. The following playlist snippets
> illustrate this point. Can you confirm if this is the expected client
> behavior?
>
>
>
> *Case #1:*
>
> #EXTINF:4.000,
>
> v1_1-7727.m4s
>
> The origin would respond by bursting the bytes it has (up to 375123) and
> then releasing the remainder as each part boundary becomes available. This
> would give it the independent part it needs to start, plus all the segments
> up to and including the HINTed part.
>
>
>
> *Case #2:*
> Origin would respond by bursting the bytes from 245668 to 375123 and then
> releasing the remainder as each part boundary becomes available.
>
>
>
> *Case #3:*
> Note that this request is NOT using the RFC8673 convention for
> last-byte-pos, because the last-byte-pos is known. The origin would respond
> by adding in a content-length response header, signaling the total bytes in
> the content-range header and bursting all these bytes since all the content
> is fully available.
>
>
>
> This is the steady state condition of the player from this point on. It
> keeps making a series of open-ended range requests against each new segment
> starting at an offset of 0. This continues until it needs to switch,
> encounters a discontinuity or playback pauses.  During this time it
> continues to update the media playlist at a period of the part duration, in
> order to maintain its knowledge of the internal structure of each segment
> and also to discover discontinuities.
>
>
>
> Cheers
>
> Will
>
>
>
>
>
> *From: *Roger Pantos <rpantos=[email-masked]>
> *Date: *Tuesday, August 25, 2020 at 12:38 PM
> *To: *"Law, Will" <[email-masked]>
> *Cc: *"[email-masked]" <[email-masked]>
> *Subject: *Re: [Hls-interest] LL-HLS: status-code expected in the
> response to the PRELOAD request
>
>
>
> Okay, I think that approach would work well enough. We’ll take a closer
> look at it once iOS 14 et al are in the can. Assuming it works out we can
> put a reference to that part of RFC 8673 into the EXT-X-PRELOAD-HINT
> section of the HLS spec.
>
>
>
> We can also update our clients to conform to that behavior, although it
> might take a while to land that in an iOS/macOS release.
>
>
>
>
>
> Roger.
>
>
>
> @Roger – re “*Note that when using H2 (or H3) that there is a fourth
> option, which is to return 206 and not supply any Content-Length header at
> all.”*
>
> This is a good point however it still leaves open the question of what
> content-range header should be returned. Per the H2 spec (snippet below),
> the requirements of HTTP/1.1 Range Requests are carried forward under
> HTTP/2. These requirements RFC7233 state that the “the server generating
> the 206 response MUST generate a Content-Range header field,” and also that
> the byte-range must contain a last-byte-pos value and that a value of * can
> be used for an unknown total length of the object but it cannot be used for
> last-byte-pos. Since the last-byte-pos is not known in this case, we have
> the same problem with H2 that we have with H1.
>
>
>
> The solution proposed by https://tools.ietf.org/html/rfc8673
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc8673&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=oMR4zfPXrho88f5S0AzpDC5z3D0bwKPKsOm2kymBHBk&e=>
>  would seem to work for H2 as well as it would for H1. Per this solution,
> the client should never make an open ended range request if it is expecting
> an aggregated response from a fixed offset. It should instead send a
> request with a very large number (9007199254740991 has been proposed in
> this thread) as the last-byte-pos in the range request. This would signal
> the server (or origin) to begin a response that starts at the requested
> offset and aggregates over time until the object is completely transferred.
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__tools.ietf.org_html_rfc7540-23section-2D8&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=BzKvOAFjcWwd1oNZnRb_NH1b50kZYcJ245JTW83T53g&e=>
>
> *From: *Roger Pantos <rpantos=[email-masked]>
> *Date: *Wednesday, August 19, 2020 at 1:17 PM
> *To: *"[email-masked]" <[email-masked]>
> *Subject: *Re: [Hls-interest] LL-HLS: status-code expected in the
> response to the PRELOAD request
>
>
>
>
>
>
>
>
> Note that when using H2 (or H3) that there is a fourth option, which is to
> return 206 and not supply any Content-Length header at all. This (I
> believe) became the expected behavior when HTTP/2 removed support for
> chunked transfer encoding:
> https://httpwg.org/specs/rfc7540.html#HttpSequence
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__httpwg.org_specs_rfc7540.html-23HttpSequence&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=QsYGd9vU3GGNCIKQjBOnvGSnXVh73OH0Kyi1KcPWpDw&e=>
>
>
>
>
>
> Roger Pantos
>
> Apple Inc.
>
>
>
>
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=R-IP9-Hj--z3sXAZZX9rokMs5qnnChNcbFO4PLft4_M&s=Yv0CtXmSurk_NBFYprpu4mN6giQ1RFARMSK8zxyR6Rg&e=>
>
>
>
>
> --
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_hls-2Dinterest&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=KkevKJerDHRF9WRs8nW8Ew&m=QUdz0dxv2aY_mBTjkxa9DgyIAgM1TzH8j_w8i_spX2w&s=VZVvigVlbA5cXwltSzsJSJ8ZrOEueLMD1jVYQO51fRk&e=>
>
>
> --