Subject: [AVT] Consensus on profile-level-id in 3984bis

From: Tom Taylor <[email-masked]>
Date: Thu, 21 May 2009 09:36:12 +0200

I find a clear consensus that level upgrade capability is required in 3984bis. I 
find a consensus that modification of 3984bis is required to provide a practical 
realization of this capability. That should simplify subsequent discussion.

Now I have a couple of questions.

1) Why does a peer care about a given implementation's sending capability?

2) The only interoperability problem I can see is that if a legacy 
implementation receives an ANSWER with a level higher than it has sent it will 
view that answer as invalid. Are there any other possible interoperability problems?

Tom Taylor

---

From: Roni Even <[email-masked]>
Date: Thu, 21 May 2009 10:53:39 +0300

Tom,
Inline
Roni

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Tom
Taylor
Sent: Thursday, May 21, 2009 10:36 AM
To: IETF AVT WG
Subject: [AVT] Consensus on profile-level-id in 3984bis

I find a clear consensus that level upgrade capability is required in
3984bis. I 
find a consensus that modification of 3984bis is required to provide a
practical 
realization of this capability. That should simplify subsequent discussion.

If the peer wants to request a specific mode from the sender, for example by
using image attribute mmusic draft.

2) The only interoperability problem I can see is that if a legacy 
implementation receives an ANSWER with a level higher than it has sent it
will 
view that answer as invalid. Are there any other possible interoperability
problems?

This is the same as level downgrade which was not allowed in RFC 3984 and
added in the bis draft. Please remember that support for a level implies
support for lower levels.

Tom Taylor
_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www.ietf.org/mailman/listinfo/avt

---

From: Ye-Kui Wang <[email-masked]>
Date: Thu, 21 May 2009 11:13:25 -0400

Another interoperability issue relates to the semantics change of the level
part of profile-level-id. Currently, the level part, when used capability
exchange or session setup, gives the upper limit of both decoding capability
and encoding capability, as seen from the following two paragraphs excerpted
from the bis draft (the same in spirit in 3984):

"The profile-level-id parameter indicates the default sub-profile, i.e. the
subset of coding tools that may have been used to generate the stream or
that the receiver supports, and the default level of the stream or the
receiver supports."

"If the profile-level-id parameter is used for capability exchange or
session setup procedure, it indicates the subset of coding tools, which is
equal to the default sub-profile, and the highest level, which is equal to
the default level, that the codec supports. All levels lower than the
default level are also supported by the codec. " 

When level upgrade is allowed, then this semantics must change, such that
the level part basically indicates only the decoding capability. This change
of course creates different understandings between the two sides whenever
one is legacy and the other is new. 

We have at least the following options to go to allow for level upgrade (the
semantics change above is anyway needed): 
- To add "level-upgrade-allowed". 
- To add "level-upgrade-allowed" and another parameter, say
sending-capability-level, to indicate the sending capability expressed as a
level value. 
- To go wildly, by making all paramters not starting with "sprop-" as
receiving capabilities. Another group of parameters not starting with
"sprop-" can be optionally used to indicate sending capabilities. When these
sending capability parameters are not present, the sending capabilities are
by default equal to the receiving capabilties. 

A decision needs to be made which one to take. 

In any case, I hope a proponent could provide relatively mature text,
including changes to semantics, offer/answer rules (including parameter sets
considerations in offer/answer), declarative usage, SDP examples, and
discussion of backward compatibility issues. 

Note that all the changes will also affect the SVC payload draft, as they
basically need to be appropriately integrated to the SVC payload draft.

BR, YK

> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Roni Even
> Sent: Thursday, May 21, 2009 3:54 AM
> To: 'Tom Taylor'; 'IETF AVT WG'
> Subject: Re: [AVT] Consensus on profile-level-id in 3984bis
> 
> Tom,
> Inline
> Roni
> 
> From: [email-masked] [mailto:[email-masked]] On 
> Behalf Of Tom Taylor
> I find a clear consensus that level upgrade capability is 
> required in 3984bis. I find a consensus that modification of 
> 3984bis is required to provide a practical realization of 
> this capability. That should simplify subsequent discussion.
> 
> If the peer wants to request a specific mode from the sender, 
> for example by using image attribute mmusic draft.
> 
> 2) The only interoperability problem I can see is that if a 
> legacy implementation receives an ANSWER with a level higher 
> than it has sent it will view that answer as invalid. Are 
> there any other possible interoperability problems?
> 
> This is the same as level downgrade which was not allowed in 
> RFC 3984 and added in the bis draft. Please remember that 
> support for a level implies support for lower levels.
> 
> 
> Tom Taylor

---

From: Tom Taylor <[email-masked]>
Date: Thu, 21 May 2009 18:01:30 +0200

It appears that the optimum outcome is to be able to signal both send and 
receive capability. The legacy approach loses some of this information, since 
profile-level-id contains the lesser of the two values. In a legacy-new 
interworking situation it seems unlikely that we can improve on this.

That suggests that profile-level-id should retain its current semantics, and one 
or two new parameters (only one of which would be present in the offer or 
answer) would indicate two things:
a) a value which is higher than the value given in profile-level-id;
b) whether that value is a send or receive capability.

If it's just one parameter, it would have to be double valued
e.g., alternate-level-id="send,<value>".

If it is two parameters, the appropriate parameter to include would depend on 
which whether the capability being reported is send or receive,
e.g., send-level-id=<value>
or    recv-level-id=<value
but never both in the same stream.

> basically need to be appropriately integrated to the SVC payload draft.
...

---

From: Ye-Kui Wang <[email-masked]>
Date: Thu, 21 May 2009 14:00:14 -0400

This should work. But the semantics profile-level-id still changes. The
level part means the same when no the new parameters are not present;
however if one, e.g., alternate-level-id="send,<value>", is present, then
the correponding capability, the sending capability in this example, is
updated by the new parameter. 

BR, YK 

> Sent: Thursday, May 21, 2009 12:02 PM
> To: IETF AVT WG
> It appears that the optimum outcome is to be able to signal 
> both send and receive capability. The legacy approach loses 
> some of this information, since profile-level-id contains the 
> lesser of the two values. In a legacy-new interworking 
> situation it seems unlikely that we can improve on this.
> 
> That suggests that profile-level-id should retain its current 
> semantics, and one or two new parameters (only one of which 
> would be present in the offer or
> If it is two parameters, the appropriate parameter to include 
> would depend on which whether the capability being reported 
> is send or receive, e.g., send-level-id=<value>
> or    recv-level-id=<value
> > Another interoperability issue relates to the semantics 
> change of the 
> > level part of profile-level-id. Currently, the level part, 
> when used 
> > capability exchange or session setup, gives the upper limit of both 
> > decoding capability and encoding capability, as seen from the 
> > following two paragraphs excerpted from the bis draft (the 
> same in spirit in 3984):
> > 
> > "The profile-level-id parameter indicates the default sub-profile, 
> > i.e. the subset of coding tools that may have been used to generate 
> > the stream or that the receiver supports, and the default 
> level of the 
> > stream or the receiver supports."
> > 
> > session setup procedure, it indicates the subset of coding tools, 
> > which is equal to the default sub-profile, and the highest level, 
> > which is equal to the default level, that the codec supports. All 
> > levels lower than the default level are also supported by 
> the codec. "
> > 
> > When level upgrade is allowed, then this semantics must 
> change, such 
> > that the level part basically indicates only the decoding 
> capability. 
> > This change of course creates different understandings 
> between the two 
> > sides whenever one is legacy and the other is new.
> > 
> > We have at least the following options to go to allow for level 
> > upgrade (the semantics change above is anyway needed):
> > sending-capability-level, to indicate the sending 
> capability expressed 
> > as a level value.
> > "sprop-" can be optionally used to indicate sending 
> capabilities. When 
> > these sending capability parameters are not present, the sending 
> > capabilities are by default equal to the receiving capabilties.
> > 
> > including changes to semantics, offer/answer rules (including 
> > parameter sets considerations in offer/answer), declarative 
> usage, SDP 
> > examples, and discussion of backward compatibility issues.
> > 
> > Note that all the changes will also affect the SVC payload 
> draft, as 
> > they basically need to be appropriately integrated to the 
> SVC payload draft.
> > 
> > BR, YK
> > 
> ...

---

From: Randell Jesup <[email-masked]>
Date: Thu, 21 May 2009 22:04:19 -0400

Tom Taylor <[email-masked]> writes:
>receive capability. The legacy approach loses some of this information,
>since profile-level-id contains the lesser of the two values. In a
>legacy-new interworking situation it seems unlikely that we can improve on
>this.

Well, I'm not sure you can say that a legacy device definitively uses the
lesser of send/receive in profile-level-id.  Because of the way
profile-level-id works, and how H.264 encodes streams, there is no problem
with currently specifying a level above what you can "truly" support
*sending*.  I.e. you can say you can receive and send level 3.0 via
profile-level-id, but nothing forces you to use the full limits of level
3.0 when sending.  You can send a stream that could be encoded identically
in level 2.0; you can even mark it as level 3.0 in the SPS/PPS NAL units.

On the other side of the equation, it is the lesser - if you can only
receive level 2.0 when all the limits in the received stream are maxed out,
you can't use anything above level 2.0 in profile-level-id.  (You can
*extend* it (such as with max-fs), if you also agree to decode the
extension as well.)

Those are the *current* semantics of profile-level-id.  A legacy device may
(or may not) as mentioned over-state it's ability to encode.

You could use an parameter that specifies the max sending level supported
in an offer (there's no point in an answer I believe).  I originally
proposed this.  However, on reflection, there's very little if anything to
gain from that that I can see.  A receiver could respond to
level-upgrade-allowed=1 (or level-asymmetry-allowed=1) with the highest
level it can receive.  Now each side knows the highest level the other can
receive, and can send up to that level.  The only downside I can possibly
think of to not knowing the maximum sending ability is that a receiver
could perhaps not allocate as many resources if it knew the offerer
couldn't send above level N.  Example:

A: Can receive level 2.0 (small LCD perhaps), can send level 3.0
B: Can receive level 5.0, can send level 3.0 (more common form of send/recv
   asymmetry)

A offers to be with profile-level-id=420014;level-asymmetry-allowed=1
B answers with profile-level-id=420032

A will send at level 3.0
B will send at level 2.0 (and must allocate resources to potentially
  receive 5.0)

If you had a maximum-send-level parameter:

A offers to be with profile-level-id=420014;sprop-max-level=1E
B answers with profile-level-id=42001E

A will send at level 3.0
B will send at level 2.0 (and only has to allocate resources for level 3.0)

The question is "is it worth it"?  Also, while this may let you limit
resources, you may also limit the options of the sender.  For example,
in the first case above, since A knows B can receive 5.0, A could send a
higher frame size than 3.0 supports (perhaps at a lower framerate), even
though A can't support sending "full" level 3.1.

I should also note that quantizing the maximum level you can send is VERY
subjective/arbitrary, because of how the encoding works.  See the
discussion above about how you don't have to max out the level you're
supposedly sending.

So my suggestion is to use level-asymmetry-allowed.  (See my previous post
for how this differs from level-upgrade-allowed - it allows asymmetric
levels on level downgrade as well, which is actually probably a more common
case.) 

>That suggests that profile-level-id should retain its current semantics,
>and one or two new parameters (only one of which would be present in the
>offer or answer) would indicate two things:
>If it is two parameters, the appropriate parameter to include would depend
>on which whether the capability being reported is send or receive,
>> semantics change above is anyway needed): - To add
>> "level-upgrade-allowed". - To add "level-upgrade-allowed" and another
>> parameter, say
>> level value. - To go wildly, by making all paramters not starting with
>> "sprop-" as

-- 
Randell Jesup, Worldgate (developers of the Ojo videophone), ex-Amiga OS team
[email-masked]
"The fetters imposed on liberty at home have ever been forged out of the weapons
provided for defence against real, pretended, or imaginary dangers from abroad."
		- James Madison, 4th US president (1751-1836)

---

From: Ye-Kui Wang <[email-masked]>
Date: Thu, 21 May 2009 22:44:54 -0400

Randell,

If the following are considered, then let the other side know the sending
capability (as well as whether level upgrade is allowed) would be very
useful. 

- One may want to request a specific mode, e.g. a spatial resolution, from
the sender, for example by using image attribute mmusic draft.
- Video adaptation, in terms of bitrate, frame rate, and/or spatial
resolution etc, may happen during the session, and may be initiated by
either side. 

BR, YK 

> Behalf Of Randell Jesup
> Sent: Thursday, May 21, 2009 10:04 PM
> To: Tom Taylor
> Cc: IETF AVT WG
> both send 
> >and receive capability. The legacy approach loses some of this 
> >information, since profile-level-id contains the lesser of the two 
> >values. In a legacy-new interworking situation it seems 
> unlikely that 
> >we can improve on this.
> 
> Well, I'm not sure you can say that a legacy device 
> definitively uses the lesser of send/receive in 
> profile-level-id.  Because of the way profile-level-id works, 
> and how H.264 encodes streams, there is no problem with 
> currently specifying a level above what you can "truly" 
> support *sending*.  I.e. you can say you can receive and send 
> level 3.0 via profile-level-id, but nothing forces you to use 
> the full limits of level 3.0 when sending.  You can send a 
> stream that could be encoded identically in level 2.0; you 
> can even mark it as level 3.0 in the SPS/PPS NAL units.
> 
> On the other side of the equation, it is the lesser - if you 
> can only receive level 2.0 when all the limits in the 
> received stream are maxed out, you can't use anything above 
> level 2.0 in profile-level-id.  (You can
> *extend* it (such as with max-fs), if you also agree to 
> decode the extension as well.)
> 
> Those are the *current* semantics of profile-level-id.  A 
> legacy device may (or may not) as mentioned over-state it's 
> ability to encode.
> 
> You could use an parameter that specifies the max sending 
> level supported in an offer (there's no point in an answer I 
> believe).  I originally proposed this.  However, on 
> reflection, there's very little if anything to gain from that 
> that I can see.  A receiver could respond to
> level-upgrade-allowed=1 (or level-asymmetry-allowed=1) with 
> the highest level it can receive.  Now each side knows the 
> highest level the other can receive, and can send up to that 
> level.  The only downside I can possibly think of to not 
> knowing the maximum sending ability is that a receiver could 
> perhaps not allocate as many resources if it knew the offerer 
> The question is "is it worth it"?  Also, while this may let 
> you limit resources, you may also limit the options of the 
> sender.  For example, in the first case above, since A knows 
> B can receive 5.0, A could send a higher frame size than 3.0 
> supports (perhaps at a lower framerate), even though A can't 
> support sending "full" level 3.1.
> 
> I should also note that quantizing the maximum level you can 
> send is VERY subjective/arbitrary, because of how the 
> encoding works.  See the discussion above about how you don't 
> have to max out the level you're supposedly sending.
> 
> So my suggestion is to use level-asymmetry-allowed.  (See my 
> previous post for how this differs from level-upgrade-allowed 
> - it allows asymmetric levels on level downgrade as well, 
> which is actually probably a more common
> case.) 
> 
> >semantics, and one or two new parameters (only one of which would be 
> >present in the offer or answer) would indicate two things:
> >If it's just one parameter, it would have to be double valued e.g., 
> >alternate-level-id="send,<value>".
> >
> >If it is two parameters, the appropriate parameter to include would 
> >depend on which whether the capability being reported is send or 
> >receive, e.g., send-level-id=<value>
> >or    recv-level-id=<value
> >> the stream or that the receiver supports, and the default level of 
> >> the stream or the receiver supports."
> >>
> >> This change of course creates different understandings between the 
> >> two sides whenever one is legacy and the other is new.
> >>
> >> upgrade (the semantics change above is anyway needed): - To add 
> >> parameter, say sending-capability-level, to indicate the sending 
> >> capability expressed as a level value. - To go wildly, by 
> making all 
> >> paramters not starting with "sprop-" as receiving capabilities. 
> >> Another group of parameters not starting with "sprop-" can be 
> >> optionally used to indicate sending capabilities. When 
> these sending 
> >> capability parameters are not present, the sending 
> capabilities are 
> >> parameter sets considerations in offer/answer), declarative usage, 
> >> SDP examples, and discussion of backward compatibility issues.
> >>
> Randell Jesup, Worldgate (developers of the Ojo videophone), 
> ex-Amiga OS team [email-masked] "The fetters imposed on 
> liberty at home have ever been forged out of the weapons

---

From: Randell Jesup <[email-masked]>
Date: Thu, 21 May 2009 23:13:45 -0400

>the sender, for example by using image attribute mmusic draft.

Does adding a max-send-level actually help you there?  How does this
interact with how you decide what your max sending level is?  As mentioned
below, choosing a value for that (other than basing it on resolution alone,
and that just might put a lower bound on it) is extremely subjective at
best.

>resolution etc, may happen during the session, and may be initiated by

Again, how would this work in practice, and how would adding it here help?
Adaptation within the negotiated limits can occur now, so I don't see the
advantage here.  If you need to change the maximum limits (why? weren't
they properly negotiated to start with?), then you can re-invite.  I think
we need a concrete example, in order to know if what you're proposing is
sufficient - or if it works.  I'm quite unclear on how you believe
specifying a max send level will help anything, since your list is quite
vague.

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 22 May 2009 10:57:46 -0400

> From: Randell Jesup [mailto:[email-masked]] 
> Sent: Thursday, May 21, 2009 11:14 PM
> To: Ye-Kui Wang
> Cc: 'Tom Taylor'; 'IETF AVT WG'
> >If the following are considered, then let the other side know the 
> >sending capability (as well as whether level upgrade is 
> allowed) would 
> >be very useful.
> >
> >- One may want to request a specific mode, e.g. a spatial 
> resolution, 
> >from the sender, for example by using image attribute mmusic draft.
> 
> Does adding a max-send-level actually help you there?  

Yes.

> How does this interact with how you decide what your max sending 
> level is?  

You don't decide your max sending level, your max sending level is your max
sending capability, a hard capability limit of you. 

> As mentioned below, choosing a value for that 
> (other than basing it on resolution alone, and that just 
> might put a lower bound on it) is extremely subjective at best.
> 

Example: If one side lets the other side know its max sending capability,
then the other side would not even try to request a specific capability that
is impossible to be satisfied. Otherwise, the other side may just send
useless requests. 

> Adaptation within the negotiated limits can occur now, so I 
> don't see the advantage here.  If you need to change the 
> maximum limits (why? weren't they properly negotiated to 
> start with?), then you can re-invite.  I think we need a 
> concrete example, in order to know if what you're proposing 
> is sufficient - or if it works.  I'm quite unclear on how you 
> believe specifying a max send level will help anything, since 
> your list is quite vague.
> 

Example

The offerer does not tell the answerer its max sending capability, which is
level B. A negotiation is done, where the answer includes level C (C>B), and
the offerer starts to send at C (actually B, but the answerer may not be
sure). After a while, the answerer senses that the connecting bandwidth is
higher, and it wants to receive at a higher bitrate, which is still within
the limit of level C, but above the limit of B, and sends a meaningless
adaption request using re-invite. 

However, if the offerer does tell the answerer its max sending capability,
such meaningless adaption requests are avoided. I believe there are more
examples, but I hope this already helps. 

BR, YK

> >> From: [email-masked] [mailto:[email-masked]] 
> On Behalf Of 
> >> Randell Jesup
> definitively uses 
> >> the lesser of send/receive in profile-level-id.  Because 
> of the way 
> >> profile-level-id works, and how H.264 encodes streams, there is no 
> >> problem with currently specifying a level above what you 
> can "truly"
> >> support *sending*.  I.e. you can say you can receive and 
> send level 
> >> 3.0 via profile-level-id, but nothing forces you to use the full 
> >> limits of level 3.0 when sending.  You can send a stream 
> that could 
> >> be encoded identically in level 2.0; you can even mark it as level 
> >> 3.0 in the SPS/PPS NAL units.
> >> 
> >> receive level 2.0 when all the limits in the received stream are 
> >> maxed out, you can't use anything above level 2.0 in 
> >> profile-level-id.  (You can
> >> Those are the *current* semantics of profile-level-id.  A legacy 
> >> device may (or may not) as mentioned over-state it's ability to 
> >> encode.
> >> 
> >> You could use an parameter that specifies the max sending level 
> >> supported in an offer (there's no point in an answer I 
> believe).  I 
> >> originally proposed this.  However, on reflection, there's very 
> >> little if anything to gain from that that I can see.  A receiver 
> >> could respond to
> >> level-upgrade-allowed=1 (or level-asymmetry-allowed=1) with the 
> >> highest level it can receive.  Now each side knows the 
> highest level 
> >> the other can receive, and can send up to that level.  The only 
> >> downside I can possibly think of to not knowing the 
> maximum sending 
> >> ability is that a receiver could perhaps not allocate as many 
> >> resources if it knew the offerer couldn't send above level N.  
> >> Example:
> >> 
> >> A offers to be with 
> profile-level-id=420014;level-asymmetry-allowed=1
> >> resources, you may also limit the options of the sender.  For 
> >> example, in the first case above, since A knows B can 
> receive 5.0, A 
> >> could send a higher frame size than 3.0 supports (perhaps 
> at a lower 
> >> framerate), even though A can't support sending "full" level 3.1.
> >> 
> >> I should also note that quantizing the maximum level you 
> can send is 
> >> VERY subjective/arbitrary, because of how the encoding works.  See 
> >> the discussion above about how you don't have to max out the level 
> >> you're supposedly sending.
> >> 
> >> So my suggestion is to use level-asymmetry-allowed.  (See 
> my previous 
> >> post for how this differs from level-upgrade-allowed
> >> - it allows asymmetric levels on level downgrade as well, which is 
> >> actually probably a more common
> >> case.)
> >> 
> between the 
> >> Randell Jesup, Worldgate (developers of the Ojo 
> videophone), ex-Amiga 
> >> OS team [email-masked] "The fetters imposed on liberty 
> at home have 
> >> ever been forged out of the weapons provided for defence against 
> >> real, pretended, or imaginary dangers from abroad."

---

From: Randell Jesup <[email-masked]>
Date: Fri, 22 May 2009 11:21:36 -0400

Ye-Kui Wang <[email-masked]> writes:
>> >sending capability (as well as whether level upgrade is allowed) would
>> >be very useful.
>> >
>> >resolution, from the sender, for example by using image attribute
>> >mmusic draft.
>> 

Max capability for what/which?  For all parameters simultaneously?
Maximum bitrate at maximum framesize at maximum macroblocks/second, with
maximum frame-to-frame motion, with maximum motion-estimation-search, etc?

I'm not saying you can't choose a maximum, but it's very, very soft - very
subjective because encoding is much more variable (and expensive) than
decoding.  It's (much) easier to estimate a worst-case for decode.

>> As mentioned below, choosing a value for that (other than basing it on
>> resolution alone, and that just might put a lower bound on it) is
>> extremely subjective at best.
>
>> Again, how would this work in practice, and how would adding it here
>> help?  Adaptation within the negotiated limits can occur now, so I don't
>> see the advantage here.  If you need to change the maximum limits (why?
>> weren't they properly negotiated to start with?), then you can
>> re-invite.  I think we need a concrete example, in order to know if what
>> you're proposing is sufficient - or if it works.  I'm quite unclear on
>> how you believe specifying a max send level will help anything, since

That's a plausible case - thanks.  However, I think you could have an
unintended result if you did that.  Lets say that the offerer uses the
"worst-case" method to decide on a max-send-level (per above).  That level
may be well below the limits it could hit when it's not worst case - for
example, it may be constrained on framesize or MBPS to level 3.0, but can
handle the outgoing bandwidth of level 4.0, and level 5.0 when other
parameters aren't worst-case.  By limiting to a max-send-level of 3.0, the
receiver won't request use of extra bandwidth or 4.0 or 5.0.  Note that 
even if you go and duplicate all the max-* parameters as sprop-max-*, you
still wouldn't be able to handle the "I can do this when other parameters
are relaxed" case.

I also already mentioned this problem below (repeated here):
>> >> example, in the first case above, since A knows B can receive 5.0, A
>> >> could send a higher frame size than 3.0 supports (perhaps at a lower

I think you can handle the adaptation issue as part of the adaptation
design, since there may be other reasons the offerer can't raise the
bandwidth (or whatever).

>> >> example, in the first case above, since A knows B can receive 5.0, A
>> >> could send a higher frame size than 3.0 supports (perhaps at a lower
>> >> I should also note that quantizing the maximum level you can send is
>> >> So my suggestion is to use level-asymmetry-allowed.  (See my previous
>> >> post for how this differs from level-upgrade-allowed - it allows
>> >> asymmetric levels on level downgrade as well, which is actually
>> >> probably a more common case.)
>> >>

---

From: Ye-Kui Wang <[email-masked]>
Date: Fri, 22 May 2009 13:10:29 -0400

I think the root of most of the disputes is, given a video encoder, whether
you can label it with a max level. In theory, I think you are right that,
given the computing resource of the encoder, it is possible to compromise
among parameters. However, in practice, when you get an encoder product
(which is typically optimized or even fixed for certain operating range), it
would typically be labelled with a profile at a level, and it should not be
common that users (of the encoder product) would try to compromise among
parameters and see whether it can produce a bitstream with a higher level,
and in many cases, the encoder product does not provide enough space for you
play around this. The level labelled by the encoder manufacture is usually
the max level. There are of course also purely PC software based encoders
can be very flexible, and can be configured to be almost any profile and
level. But once it is configured to be at a profile and a level and the
encoder is initialized, then that level should be used as the max level. 
 
BR, YK

> Sent: Friday, May 22, 2009 11:22 AM
> To: Ye-Kui Wang
> >> >sending capability (as well as whether level upgrade is allowed) 
> >> >would be very useful.
> >> >
> >You don't decide your max sending level, your max sending 
> level is your 
> >max sending capability, a hard capability limit of you.
> 
> Maximum bitrate at maximum framesize at maximum 
> macroblocks/second, with maximum frame-to-frame motion, with 
> maximum motion-estimation-search, etc?
> 
> I'm not saying you can't choose a maximum, but it's very, 
> very soft - very subjective because encoding is much more 
> variable (and expensive) than decoding.  It's (much) easier 
> to estimate a worst-case for decode.
> 
> >> As mentioned below, choosing a value for that (other than 
> basing it 
> >> on resolution alone, and that just might put a lower bound 
> on it) is 
> >Example: If one side lets the other side know its max sending 
> >capability, then the other side would not even try to request a 
> >specific capability that is impossible to be satisfied. 
> Otherwise, the 
> >other side may just send useless requests.
> >
> >> >resolution etc, may happen during the session, and may be 
> initiated 
> >> >by either side.
> >> 
> >> help?  Adaptation within the negotiated limits can occur now, so I 
> maximum limits (why?
> >> re-invite.  I think we need a concrete example, in order 
> to know if 
> >> what you're proposing is sufficient - or if it works.  I'm quite 
> >> unclear on how you believe specifying a max send level will help 
> >> anything, since your list is quite vague.
> >> 
> >
> >Example
> >
> >The offerer does not tell the answerer its max sending capability, 
> >which is level B. A negotiation is done, where the answer includes 
> >level C (C>B), and the offerer starts to send at C (actually 
> B, but the 
> >answerer may not be sure). After a while, the answerer 
> senses that the 
> >connecting bandwidth is higher, and it wants to receive at a higher 
> >bitrate, which is still within the limit of level C, but above the 
> >limit of B, and sends a meaningless adaption request using re-invite.
> >
> >However, if the offerer does tell the answerer its max sending 
> >capability, such meaningless adaption requests are avoided. 
> I believe 
> >there are more examples, but I hope this already helps.
> 
> That's a plausible case - thanks.  However, I think you could 
> have an unintended result if you did that.  Lets say that the 
> offerer uses the "worst-case" method to decide on a 
> max-send-level (per above).  That level may be well below the 
> limits it could hit when it's not worst case - for example, 
> it may be constrained on framesize or MBPS to level 3.0, but 
> can handle the outgoing bandwidth of level 4.0, and level 5.0 
> when other parameters aren't worst-case.  By limiting to a 
> max-send-level of 3.0, the receiver won't request use of 
> extra bandwidth or 4.0 or 5.0.  Note that even if you go and 
> duplicate all the max-* parameters as sprop-max-*, you still 
> wouldn't be able to handle the "I can do this when other 
> parameters are relaxed" case.
> 
> >> >> The question is "is it worth it"?  Also, while this may let you 
> >> >> limit resources, you may also limit the options of the sender.  
> >> >> For example, in the first case above, since A knows B 
> can receive 
> >> >> 5.0, A could send a higher frame size than 3.0 supports 
> (perhaps 
> >> >> at a lower framerate), even though A can't support 
> sending "full" level 3.1.
> 
> I think you can handle the adaptation issue as part of the 
> adaptation design, since there may be other reasons the 
> offerer can't raise the bandwidth (or whatever).
> 
> >BR, YK
> >
> >
> >> >BR, YK
> >> >
> >> >> B: Can receive level 5.0, can send level 3.0 (more 
> common form of 
> >> >> send/recv
> >> >>    asymmetry)
> >> >> 
> >> >> A offers to be with
> >> >> B will send at level 2.0 (and only has to allocate 
> resources for 
> >> >> level 3.0)
> >> >> 
> >> >> 
> >> >> The question is "is it worth it"?  Also, while this may let you 
> >> >> limit resources, you may also limit the options of the sender.  
> >> >> For example, in the first case above, since A knows B 
> can receive 
> >> >> 5.0, A could send a higher frame size than 3.0 supports 
> (perhaps 
> >> >> at a lower framerate), even though A can't support 
> sending "full" level 3.1.
> >> >> 
> >> >> I should also note that quantizing the maximum level 
> you can send 
> >> >> is VERY subjective/arbitrary, because of how the 
> encoding works.  
> >> >> See the discussion above about how you don't have to 
> max out the 
> >> >> level you're supposedly sending.
> >> >> 
> >> >> previous post for how this differs from 
> level-upgrade-allowed - it 
> >> >> allows asymmetric levels on level downgrade as well, which is 
> >> >> actually probably a more common case.)
> >> >> 
> >> >> >Audio/Video Transport Working Group [email-masked] 
> >> >> Audio/Video Transport Working Group [email-masked] 
> ex-Amiga OS team

---

From: Roni Even <[email-masked]>
Date: Fri, 22 May 2009 23:55:25 +0300

YK,
I think you are right that we are now to the root of the discussion. The
levels in H.264 annex A specify some compromise in order to allow mass
production devices like set top boxes use fixed architecture for example in
order to define the frame buffer size on the device and typically will not
negotiate the level (mostly streaming applications). On the other side video
conferencing systems have much more flexibility and control over the encoder
since they are part of a system that considers also the display and network
connection when they offer a specific level (which may be lower than the
maximum that they can support). Having more control on the encoder allows
them to get requests from the remote system to send pictures at specific
resolutions and aspect ratios. I think I mentioned before that the offer can
offer a low level and add capabilities using the optional parameters, this
working point will be between two specified levels from table A-1, higher
than the one in the offer but lower than the one above it. I think that in
this case if the answer will include the higher level this will still work,
and since the encoder is flexible it may take advantage of the new
information.

Roni 

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Ye-Kui
Wang
Sent: Friday, May 22, 2009 8:10 PM
To: 'Randell Jesup'
Cc: 'Tom Taylor'; 'IETF AVT WG'
Subject: Re: [AVT] Consensus on profile-level-id in 3984bis

BR, YK

---

From: Ye-Kui Wang <[email-masked]>
Date: Sat, 23 May 2009 12:08:14 -0400

Roni,

> I think I mentioned before that the offer can 
> offer a low level and add capabilities using the optional 
> parameters, this working point will be between two specified 
> levels from table A-1, higher than the one in the offer but 
> lower than the one above it. I think that in this case if the 
> answer will include the higher level this will still work, 
> and since the encoder is flexible it may take advantage of 
> the new information.

Those max-* parameters may only be used to indicate receiver capabilities
not anything else. That is, an offerer or answerer may add extra
capabilities than the included level for receiving capabilities, but not for
sending. 

BR, YK

> From: Roni Even [mailto:[email-masked]] 
> Sent: Friday, May 22, 2009 4:55 PM
> To: 'Ye-Kui Wang'; 'Randell Jesup'
> Subject: RE: [AVT] Consensus on profile-level-id in 3984bis
> 
> YK,
> I think you are right that we are now to the root of the 
> discussion. The levels in H.264 annex A specify some 
> compromise in order to allow mass production devices like set 
> top boxes use fixed architecture for example in order to 
> define the frame buffer size on the device and typically will 
> not negotiate the level (mostly streaming applications). On 
> the other side video conferencing systems have much more 
> flexibility and control over the encoder since they are part 
> of a system that considers also the display and network 
> connection when they offer a specific level (which may be 
> lower than the maximum that they can support). Having more 
> control on the encoder allows them to get requests from the 
> remote system to send pictures at specific resolutions and 
> aspect ratios. I think I mentioned before that the offer can 
> offer a low level and add capabilities using the optional 
> parameters, this working point will be between two specified 
> levels from table A-1, higher than the one in the offer but 
> lower than the one above it. I think that in this case if the 
> answer will include the higher level this will still work, 
> and since the encoder is flexible it may take advantage of 
> the new information.
> 
> Roni 
> 
> Behalf Of Ye-Kui Wang
> I think the root of most of the disputes is, given a video 
> encoder, whether you can label it with a max level. In 
> theory, I think you are right that, given the computing 
> resource of the encoder, it is possible to compromise among 
> parameters. However, in practice, when you get an encoder 
> product (which is typically optimized or even fixed for 
> certain operating range), it would typically be labelled with 
> a profile at a level, and it should not be common that users 
> (of the encoder product) would try to compromise among 
> parameters and see whether it can produce a bitstream with a 
> higher level, and in many cases, the encoder product does not 
> provide enough space for you play around this. The level 
> labelled by the encoder manufacture is usually the max level. 
> There are of course also purely PC software based encoders 
> can be very flexible, and can be configured to be almost any 
> profile and level. But once it is configured to be at a 
> profile and a level and the encoder is initialized, then that 
> level should be used as the max level. 
>  
> BR, YK
> 
> allowed) 
> > Maximum bitrate at maximum framesize at maximum macroblocks/second, 
> > with maximum frame-to-frame motion, with maximum 
> > motion-estimation-search, etc?
> > 
> very soft - 
> > very subjective because encoding is much more variable (and 
> expensive) 
> > than decoding.  It's (much) easier to estimate a worst-case for 
> > decode.
> > 
> > "worst-case" method to decide on a max-send-level (per 
> above).  That 
> > level may be well below the limits it could hit when it's not worst 
> > case - for example, it may be constrained on framesize or MBPS to 
> > level 3.0, but can handle the outgoing bandwidth of level 4.0, and 
> > level 5.0 when other parameters aren't worst-case.  By 
> limiting to a 
> > max-send-level of 3.0, the receiver won't request use of extra 
> > bandwidth or 4.0 or 5.0.  Note that even if you go and 
> duplicate all 
> > the max-* parameters as sprop-max-*, you still wouldn't be able to 
> > handle the "I can do this when other parameters are relaxed" case.
> > 
> adaptation 
> > OS team [email-masked] "The fetters imposed on liberty at 
> home have 
> > ever been forged out of the weapons provided for defence 
> against real, 
> > pretended, or imaginary dangers from abroad."

---

From: Ye-Kui Wang <[email-masked]>
Date: Wed, 27 May 2009 16:38:20 -0400

Can we conclude the consensus on this issue to be something similar as Tom
suggested?

BR, YK

> Sent: Saturday, May 23, 2009 12:08 PM
> To: 'Roni Even'; 'Randell Jesup'
> > I think I mentioned before that the offer can offer a low level and 
> > add capabilities using the optional parameters, this working point 
> > will be between two specified levels from table A-1, higher 
> than the 
> > one in the offer but lower than the one above it. I think 
> that in this 
> > case if the answer will include the higher level this will 
> still work, 
> Those max-* parameters may only be used to indicate receiver 
> capabilities not anything else. That is, an offerer or 
> answerer may add extra capabilities than the included level 
> for receiving capabilities, but not for sending. 
> 
> BR, YK
> 
> discussion. 
> > The levels in H.264 annex A specify some compromise in 
> order to allow 
> > mass production devices like set top boxes use fixed 
> architecture for 
> > example in order to define the frame buffer size on the device and 
> > typically will not negotiate the level (mostly streaming 
> > applications). On the other side video conferencing systems 
> have much 
> > more flexibility and control over the encoder since they 
> are part of a 
> > system that considers also the display and network connection when 
> > they offer a specific level (which may be lower than the 
> maximum that 
> > they can support). Having more control on the encoder 
> allows them to 
> > get requests from the remote system to send pictures at specific 
> > resolutions and aspect ratios. I think I mentioned before that the 
> > offer can offer a low level and add capabilities using the optional 
> > parameters, this working point will be between two specified levels 
> > from table A-1, higher than the one in the offer but lower than the 
> > one above it. I think that in this case if the answer will 
> include the 
> > higher level this will still work, and since the encoder is 
> flexible 
> > it may take advantage of the new information.
> > 
> > Roni
> > 
> Behalf Of 
> > Ye-Kui Wang
> > I think the root of most of the disputes is, given a video encoder, 
> > whether you can label it with a max level. In theory, I 
> think you are 
> > right that, given the computing resource of the encoder, it is 
> > possible to compromise among parameters. However, in practice, when 
> > you get an encoder product (which is typically optimized or 
> even fixed 
> > for certain operating range), it would typically be labelled with a 
> > profile at a level, and it should not be common that users (of the 
> > encoder product) would try to compromise among parameters and see 
> > whether it can produce a bitstream with a higher level, and in many 
> > cases, the encoder product does not provide enough space 
> for you play 
> > around this. The level labelled by the encoder manufacture 
> is usually 
> > the max level.
> can be very 
> > flexible, and can be configured to be almost any profile and level. 
> > But once it is configured to be at a profile and a level and the 
> macroblocks/second, 
> profile-level-id=420014;sprop-max-level=1E
> > > >> >> >> i.e. the subset of coding tools that may have 
> been used to
> > > >> >> generate

---

From: Ye-Kui Wang <[email-masked]>
Date: Thu, 28 May 2009 10:36:35 -0400

Chairs,

Please guide how to proceed. We need to progress this work, which is
dependent on by the SVC draft, which is needed by some other standards
organizations like DVB and ATSC, though it seems that none of the proponents
is going to provide detailed spec text. 

BR, YK

> From: Ye-Kui Wang [mailto:[email-masked]] 
> Sent: Wednesday, May 27, 2009 4:38 PM
> To: 'IETF AVT WG'
> Can we conclude the consensus on this issue to be something 
> similar as Tom suggested?
> 
> BR, YK
> 
> 
> > capabilities not anything else. That is, an offerer or answerer may 
> > add extra capabilities than the included level for receiving 
> > capabilities, but not for sending.
> > 
> > BR, YK
> > 
> > > > >> >> >> "If the profile-level-id parameter is used for 
> capability
> > > > >> >> exchange or
> > > > >> >> >> "level-upgrade-allowed". - To add 
> "level-upgrade-allowed"
> > > > >> >> and another
> > > > >> >> >> In any case, I hope a proponent could provide 
> relatively
> > > > >> >> mature text,

---

From: Randell Jesup <[email-masked]>
Date: Fri, 29 May 2009 08:32:04 -0400

>is going to provide detailed spec text. 

I'm willing to take a shot at some text over the weekend, modulo time with
my daughter.