Subject: [AVT] Requirements draft on video control

From: "Andrea Basso" <[email-masked]>
Date: Mon, 7 Jul 2003 17:34:10 -0400

Attached you will find draft-basso-avt-videoconreq-00.txt

Abstract:

A variety of communication services such as video
conferencing and video messaging rely on the capability of video
encoders and decoders to exchange control commands. This document
outlines this set of commands as well as the requirements for
their transport.

Comments are more than welcome.

Andrea Basso
NMS Communications
(732) 936-2118
(See attached file: draft-basso-avt-videoconreq-00.zip)

---

From: "Gary Sullivan" <[email-masked]>
Date: Tue, 8 Jul 2003 17:03:29 -0700

Much of the content of this ID will look familiar to those with
experience with ITU-T systems control specifications (such as H.245).

Some of it seems like it could use a bit of updating in that regard
(e.g., videofastupdateGOB).
Draft Recommendation H.241 is one good place to look for such
information.

Section 3 is a bit confused in its terminology.  For example it seems to
say that intra frames are also known as inter frames, which is
definitely not the case.  It mixes the terms "predictive" and
"prediction" and seems to consider a "reference" frame to be synonymous
with an intra frame.  The terms "picture" and "frame" are also kind of
mixed up.

A more consistent terminology would be as follows:

An intra picture is a picture that can be decoded without first decoding
any other picture.

A predicted (or non-intra) picture is a picture that may require data
from one or more previously-decoded pictures in order to be properly
decoded.

A reference picture is a picture that is stored in the decoder for use
as a reference in the decoding process of some subsequent picture in the
bitstream.

A non-reference picture is a picture that is not used as a reference for
the decoding process of any other picture in the bitstream.

The concepts of intra versus non-Intra and reference versus
non-reference are independent.  A particular picture can (in general) be
any one of the four types (intra reference, non-intra reference, intra
non-reference, non-intra non-reference).

A "fast update", also known as an "instantaneous decoder refresh",
involves sending an intra picture and thereafter refraining from using
any picture sent prior to that intra picture as a reference for the
decoding process of any subsequent picture sent in the stream.

In particular it is worth noting that some concepts that applied to some
older codecs are not really valid in general and should be avoided
(e.g., that a non-reference picture is called a B picture or that an
intra picture is the same thing as a fast update or that an intra
picture is always a reference picture).

The characterization of a "slice" or "GOB" as being a "stripe" is not
really quite right.  For example, a single slice in MPEG-1, H.263 Annex
O, or MPEG-4 can consist of a few macroblocks at the right edge of one
row and a few macroblocks at the left edge of the next row (i.e., two
distinct rectangular regions quite far apart from each other in the
picture can be in a single slice).  A GOB in H.261 CIF consists of THREE
partial rows of macroblocks extending only halfway across a picture.
And in H.264/AVC, a slice can be ANY collection of macroblocks scattered
around absolutely anywhere in the picture (by use of a feature known as
a "slice group map").

In regard to section 5.1 VideoFreezePicture, I believe H.262 (also known
as MPEG-2 Video) also supports a freeze picture release within the ITU-T
extension.

It is also possible to obtain random access recovery without a fast
update.  This is sometimes called "gradual decoder refresh".  See for
example the recovery point SEI message in H.264/AVC.

Best Regards,

Gary Sullivan

+> -----Original Message-----
+> From: [email-masked] [mailto:[email-masked]] On 
+> Behalf Of Andrea Basso
+> Sent: Monday, July 07, 2003 2:34 PM
+> To: [email-masked]; [email-masked]; [email-masked]
+> Cc: [email-masked]; [email-masked]; 
+> [email-masked]; [email-masked]; Orit 
+> Levin; Roni Even; [email-masked]
+> Subject: [AVT] Requirements draft on video control
+> 
+> 
+> Attached you will find draft-basso-avt-videoconreq-00.txt
+> 
+> Abstract:
+> 
+> A variety of communication services such as video
+> conferencing and video messaging rely on the capability of video
+> encoders and decoders to exchange control commands. This document
+> outlines this set of commands as well as the requirements for
+> their transport.
+> 
+> Comments are more than welcome.
+> 
+> 
+> 
+> 
+> 
+> Andrea Basso
+> NMS Communications
+> (732) 936-2118
+> (See attached file: draft-basso-avt-videoconreq-00.zip)
+> 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: "Even, Roni" <[email-masked]>
Date: Wed, 9 Jul 2003 08:46:19 +0300

Hi Gary

About
"It is also possible to obtain random access recovery without a fast
update.  This is sometimes called "gradual decoder refresh".  See for
example the recovery point SEI message in H.264/AVC."

In case of a video switch in a multipoint conference there is a need to get
a new recovery point since the decoder will receive a stream without a
previous recovery point. The procedure in H.241 allows for either a full
picture at once or gradual recovery based on an SEI recovery point message.
Both method are fine as an encoder response to a fast update request.

Regards
Roni Even

-----Original Message-----
From: Gary Sullivan [mailto:[email-masked]]
Sent: Wednesday, July 09, 2003 3:03 AM
To: Andrea Basso; [email-masked]; [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]; [email-masked];
[email-masked]; Orit Levin; Roni Even; [email-masked]
Subject: RE: [AVT] Requirements draft on video control

Best Regards,

Gary Sullivan

---

From: "Gary Sullivan" <[email-masked]>
Date: Wed, 9 Jul 2003 13:39:46 -0700

Andrea et al,

Perhaps I was having some trouble understanding the purpose of this ID.
I thought it was some sort of draft of a formal specification, rather
than just an expression of needs to be taken as input by the group.  So
I criticized the language more heavily than necessary.

In regard to the "intra frames" = "inter frames" mix-up:

+> >intra frames are also known as inter frames, which is
+> >definitely not the case
+> 
+>  I do not think the text says this.  Here is the sentence:
+> 
+>      Current coding schemes such as H.261 [2], H.263 [3], MPEG-1, 2,4
+>       [5], H.264 [6] can encode video pictures as reference 
+> frames, also
+>       known as intra frames or predicted frames, also known as inter
+>       frames.

It seems like what we have is a scoping problem.  To what set of words
does the first "also known as" phrase apply?  To me, when you said "A,
also known as B or C, also known as D", I interpreted this as saying
that A, B, C, and D are all synonyms (with the first "also known as"
applying to the "B or C").  Apparently that's not what you meant.

Also, the way the term "reference frame" is used in this document may
have contributed to my confusion.  As I noted in my original message,
that term is used differently here than it is in some other video coding
discussions.

Best Regards,

Gary Sullivan

+> -----Original Message-----
+> From: Andrea Basso [mailto:[email-masked]] 
+> Sent: Wednesday, July 09, 2003 12:17 PM
+> To: Gary Sullivan
+> Cc: Andrea Basso; [email-masked]; [email-masked]; 
+> [email-masked]; [email-masked]; 
+> [email-masked]; [email-masked]; 
+> [email-masked]; Orit Levin; Roni Even
+> Subject: RE: [AVT] Requirements draft on video control
+> 
+> 
+> 
+> Gary,
+> here are some comments.
+> 
+> 
+> > Some of it seems like it could use a bit of updating in that regard
+> > (e.g., videofastupdateGOB).
+> > Draft Recommendation H.241 is one good place to look for such
+> > information.
+> 
+> H.241 is very focused on H.264 and in very early stages in addressing
+> Commands and Indications applicable to all codecs.
+>  If you look at the H.241 draft section 6 says:
+> 
+> 
+> 
+> 6      Commands and Indications
+> 
+> 
+> 
+> 6.1    C&I applicable to all video codecs
+> 
+> For further study.
+> 
+> 
+> We plan to monitor the evolution the H.241 and to update the 
+> ID accordingly
+> if needed.
+> 
+> 
+> >intra frames are also known as inter frames, which is
+> >definitely not the case
+> 
+>  I do not think the text says this.  Here is the sentence:
+> 
+>      Current coding schemes such as H.261 [2], H.263 [3], MPEG-1, 2,4
+>       [5], H.264 [6] can encode video pictures as reference 
+> frames, also
+>       known as intra frames or predicted frames, also known as inter
+>       frames.
+> 
+> 
+> >The terms "picture" and "frame" are also kind of
+> >mixed up.
+> 
+> For progressive video, a picture is identical to a frame. 
+> The difference
+> appears for  interlaced video only.
+> The new text will use the term  'picture' only.
+> 
+> 
+> >The characterization of a "slice" or "GOB" as being a 
+> "stripe" is not
+> >really quite right.
+> 
+> The text does not say "image stripe" says "stripe" and in 
+> particular it
+> does not imply that such stripe goes border-to-border in the 
+> image (i.e.
+> can cover a partial macroblock row) nor that is should only include a
+> single  macroblock row (i.e., can cover multiple partial 
+> rows as in H.261).
+> Clarifications will be addressed in the new version of the draft.
+> 
+> 
+> -A
+> 
+> Andrea Basso
+> NMS Communications
+> (732) 936-2118
+> 
+> 
+> 
+>                                                              
+>                                                                   
+>                       "Gary Sullivan"                        
+>                                                                   
+>                       <[email-masked]        To:       
+> "Andrea Basso" <[email-masked]>, <[email-masked]>,           
+>                       rosoft.com>                   
+> <[email-masked]>, <[email-masked]>                      
+>               
+>                                                    cc:       
+> <[email-masked]>, <[email-masked]>,                         
+>                       07/08/2003 08:03 PM           
+> <[email-masked]>, <[email-masked]>, "Orit 
+> Levin"        
+>                                                     
+> <[email-masked]>, "Roni Even" <[email-masked]>, 
+>               
+>                                                     
+> <[email-masked]>                                          
+>               
+>                                                    Subject:  
+> RE: [AVT] Requirements draft on video control                     
+>                                                              
+>                                                                   
+> 
+> 
+> 
+> 
+> 
+> Much of the content of this ID will look familiar to those with
+> experience with ITU-T systems control specifications (such as H.245).
+> 
+> Some of it seems like it could use a bit of updating in that regard
+> (e.g., videofastupdateGOB).
+> Draft Recommendation H.241 is one good place to look for such
+> information.
+> 
+> Section 3 is a bit confused in its terminology.  For example 
+> it seems to
+> say that intra frames are also known as inter frames, which is
+> definitely not the case.  It mixes the terms "predictive" and
+> "prediction" and seems to consider a "reference" frame to be 
+> synonymous
+> with an intra frame.  The terms "picture" and "frame" are 
+> also kind of
+> mixed up.
+> 
+> A more consistent terminology would be as follows:
+> 
+> An intra picture is a picture that can be decoded without 
+> first decoding
+> any other picture.
+> 
+> A predicted (or non-intra) picture is a picture that may require data
+> from one or more previously-decoded pictures in order to be properly
+> decoded.
+> 
+> A reference picture is a picture that is stored in the 
+> decoder for use
+> as a reference in the decoding process of some subsequent 
+> picture in the
+> bitstream.
+> 
+> A non-reference picture is a picture that is not used as a 
+> reference for
+> the decoding process of any other picture in the bitstream.
+> 
+> The concepts of intra versus non-Intra and reference versus
+> non-reference are independent.  A particular picture can (in 
+> general) be
+> any one of the four types (intra reference, non-intra 
+> reference, intra
+> non-reference, non-intra non-reference).
+> 
+> A "fast update", also known as an "instantaneous decoder refresh",
+> involves sending an intra picture and thereafter refraining 
+> from using
+> any picture sent prior to that intra picture as a reference for the
+> decoding process of any subsequent picture sent in the stream.
+> 
+> In particular it is worth noting that some concepts that 
+> applied to some
+> older codecs are not really valid in general and should be avoided
+> (e.g., that a non-reference picture is called a B picture or that an
+> intra picture is the same thing as a fast update or that an intra
+> picture is always a reference picture).
+> 
+> The characterization of a "slice" or "GOB" as being a "stripe" is not
+> really quite right.  For example, a single slice in MPEG-1, 
+> H.263 Annex
+> O, or MPEG-4 can consist of a few macroblocks at the right 
+> edge of one
+> row and a few macroblocks at the left edge of the next row (i.e., two
+> distinct rectangular regions quite far apart from each other in the
+> picture can be in a single slice).  A GOB in H.261 CIF 
+> consists of THREE
+> partial rows of macroblocks extending only halfway across a picture.
+> And in H.264/AVC, a slice can be ANY collection of 
+> macroblocks scattered
+> around absolutely anywhere in the picture (by use of a 
+> feature known as
+> a "slice group map").
+> 
+> In regard to section 5.1 VideoFreezePicture, I believe H.262 
+> (also known
+> as MPEG-2 Video) also supports a freeze picture release 
+> within the ITU-T
+> extension.
+> 
+> It is also possible to obtain random access recovery without a fast
+> update.  This is sometimes called "gradual decoder refresh".  See for
+> example the recovery point SEI message in H.264/AVC.
+> 
+> Best Regards,
+> 
+> Gary Sullivan
+> 
+> 
+> +> -----Original Message-----
+> +> From: [email-masked] [mailto:[email-masked]] On
+> +> Behalf Of Andrea Basso
+> +> Sent: Monday, July 07, 2003 2:34 PM
+> +> To: [email-masked]; [email-masked]; [email-masked]
+> +> Cc: [email-masked]; [email-masked];
+> +> [email-masked]; [email-masked]; Orit
+> +> Levin; Roni Even; [email-masked]
+> +> Subject: [AVT] Requirements draft on video control
+> +>
+> +>
+> +> Attached you will find draft-basso-avt-videoconreq-00.txt
+> +>
+> +> Abstract:
+> +>
+> +> A variety of communication services such as video
+> +> conferencing and video messaging rely on the capability of video
+> +> encoders and decoders to exchange control commands. This document
+> +> outlines this set of commands as well as the requirements for
+> +> their transport.
+> +>
+> +> Comments are more than welcome.
+> +>
+> +>
+> +>
+> +>
+> +>
+> +> Andrea Basso
+> +> NMS Communications
+> +> (732) 936-2118
+> +> (See attached file: draft-basso-avt-videoconreq-00.zip)
+> +>
+> 
+> 
+> 
+> 
+> 
+>

---

From: Nermeen Ismail <[email-masked]>
Date: Wed, 09 Jul 2003 16:25:05 -0700

Gary,

Thanks for the comments. Definitely we need to add consistency in the terms 
we are using in the draft.

>I criticized the language more heavily than necessary.

So now that this is clarified we need to discuss the requirements that are 
specified in that draft
and see if people think that these are legitimate requirements that need to 
be addressed or not.

nermeen