Subject: Re: [Hls-interest] coming to Apple HLS clients later this year: M-JPEG support

From: "Galluccio, Robert (NBCUniversal)" <[email-masked]>
Date: Wed, 2 Mar 2022 21:20:24 +0000

Hi Roger,

Is there any consideration for introducing grids of multiple thumbnails expressed in one M-JPEG segment and therefore taking up a larger EXTINF value?

This would be similar to DASH-IF-IOP 11.5:
https://dashif-documents.azurewebsites.net/DASH-IF-IOP/master/DASH-IF-IOP.html#codecs-thumbnails

And also to the custom tags defined on this Roku developer documentation page:
https://developer.roku.com/en-gb/docs/developer-program/media-playback/trick-mode/hls-and-dash.md#ext-x-tiles

This can provide network round trip savings for VOD assets where the entire timeline is already defined when loading the asset.

And separately, with regards to Apple HLS clients support, will there be a plan to expose obtaining the thumbnail images outside of AVKit (e.g. asking the system for a thumbnail image for a given playback time)?

Best,
Rob

---

From: Roger Pantos <[email-masked]>
Date: Wed, 02 Mar 2022 13:31:03 -0800

> On Mar 2, 2022, at 1:20 PM, Galluccio, Robert (NBCUniversal) <[email-masked]> wrote:
> 
> Hi Roger,
>  
> https://dashif-documents.azurewebsites.net/DASH-IF-IOP/master/DASH-IF-IOP.html#codecs-thumbnails <https://dashif-documents.azurewebsites.net/DASH-IF-IOP/master/DASH-IF-IOP.html#codecs-thumbnails>
>  
> https://developer.roku.com/en-gb/docs/developer-program/media-playback/trick-mode/hls-and-dash.md#ext-x-tiles <https://developer.roku.com/en-gb/docs/developer-program/media-playback/trick-mode/hls-and-dash.md#ext-x-tiles>
>  

For VOD content it is pretty common to compose multiple fMP4 segments into a single MPEG-4 resource using byte range addressing. That allows clients to download all thumbnails using a single HTTP request (or to download them progressively as needed, if the client prefers).

> And separately, with regards to Apple HLS clients support, will there be a plan to expose obtaining the thumbnail images outside of AVKit (e.g. asking the system for a thumbnail image for a given p…

Yes, this is already available (for unencrypted thumbnails) using the AVAssetImageGenerator API.

Roger.