Subject: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02

From: Kevin Gross <[email-masked]>
Date: Mon, 12 Dec 2011 13:23:06 -0700

Here are some additional comments on the draft. Most of these are
suggestions for minor improvements. Items 6 and 7 represent significant
technical issues and may require discussion.

1/ Chance all occurrences of "play-out" to "playout".

2/ First line of 1.2 "Currently, most _interactive_ multimedia applications
make use of RTP and RTCP." Streaming applications commonly use HTTP.

3/ Last paragraph in 1.2 As information on RTP packet arrival times and
play-out times can be considered reception quality feedback information,
RTCP _is the appropriate conduit for delivery and playout delay information
for IDMS._" Convert from proposal to standards language.

4/ Second paragraph, section 1.3. Delete "one for the RTCP XR block type
and one for the new RTCP packet type." SDP parameters make the system work
and are not closely associated with these protocols.

5/ Additional paragraph to append to 1.4, "The ETSI TISPAN protocol differs
in that RTCP XR block are used to convey playout and delivery delays as
well as to send synchronization parameters to receivers. In this document,
a new RTCP packet type is defined to convey high-resolution synchronization
parameters to receivers.

6/ Packet Received NTP timestamp and Packet Received RTP timestamp
definitions in section 6 and Packet Received NTP timestamp definition in
section 7. What IDMS really wants to measure is when all data for the
indicated RTP timestamp is available to the rendering application. We'd
therefore like to know the arrival time of the last octet of the last
packet associated with this RTP timestamp. There is difficulty reliably
identifying the last packet. We could defining this as the last packet
before media clock advances but this does not appreciate out-of-order
generation and reordering by the network.

7/ In Packet Presented NTP timestamp definition in section 6, Clarify that
presentation time does not include any intentionally introduced delay.
Discuss presentation estimates. Suggested replacement for the
first sentence of the definition: "This timestamp reflects the wall clock
time at the earliest moment the data contained in the first octet of the
associated RTP packet could be presented to the user if no IDMS-directed
delay were introduced. In some cases, due to additional delays in external
equipment or sound propagation time through air, actual presentation time
to the user may not be accurately knowable. For a comfortable media
experience, it is preferable to delay audio with respect to video than
vise-versa. Pesimistic (longer) assumptions about playout delay should be
used for video and optimistic (shorter) ones for audio."

8/ Clarify use of 0 value in Packet Presented NTP timestamp definition in
section 6. If _no presentation timestamp is available_ then is shall be set
to 0.

9/  Packet Presented NTP timestamp in section 7. "This timestamp reflects
the wall clock time at the reference client at the moment the data
contained in the first octet of the associated RTP packet _should be
presented_ to the user. Change to directive language.

10/ SyncGroupID definition in sections 9 and 10. Remove "in network byte
order" qualifier. This qualifier may cause confusion, is not necessary and
is not applied to attribute definitions in RFC 4566.

11/ Editorial correction and clarification in section 13: "Normally,
the processing and play-out of the same media stream by _similar_ receivers
will take roughly the same amount of time. By synchronizing on packet
arrival times, you may _lose_ some accuracy, but it will be adequate for
many applications,

Kevin Gross
+1-303-447-0517
Media Network Consultant
AVA Networks - www.AVAnw.com <http://www.avanw.com/>, www.X192.org

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Thu, 15 Dec 2011 08:56:11 +0000

Hi Kevin,

Thanks for your extensive comments. My comments inline.

From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin Gross
Sent: maandag 12 december 2011 21:23
To: [email-masked]
Subject: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02

Here are some additional comments on the draft. Most of these are suggestions for minor improvements. Items 6 and 7 represent significant technical issues and may require discussion.

Ok

2/ First line of 1.2 "Currently, most _interactive_ multimedia applications make use of RTP and RTCP." Streaming applications commonly use HTTP.

I agree.

3/ Last paragraph in 1.2 As information on RTP packet arrival times and play-out times can be considered reception quality feedback information, RTCP _is the appropriate conduit for delivery and playout delay information for IDMS._" Convert from proposal to standards language.

Ok.

4/ Second paragraph, section 1.3. Delete "one for the RTCP XR block type and one for the new RTCP packet type." SDP parameters make the system work and are not closely associated with these protocols.

I'm not sure what you mean here? The sentence is meant to indicate that this draft introduces two sets of SDP parameters: one for the new XR block type, and one for the new RTCP Packet Type (sections 9 and 10 respectively)

5/ Additional paragraph to append to 1.4, "The ETSI TISPAN protocol differs in that RTCP XR block are used to convey playout and delivery delays as well as to send synchronization parameters to receivers. In this document, a new RTCP packet type is defined to convey high-resolution synchronization parameters to receivers.

Ok.

6/ Packet Received NTP timestamp and Packet Received RTP timestamp definitions in section 6 and Packet Received NTP timestamp definition in section 7. What IDMS really wants to measure is when all data for the indicated RTP timestamp is available to the rendering application. We'd therefore like to know the arrival time of the last octet of the last packet associated with this RTP timestamp. There is difficulty reliably identifying the last packet. We could defining this as the last packet before media clock advances but this does not appreciate out-of-order generation and reordering by the network.

I'm not sure whether there is a functional difference between indicating the arrival of the first packet or the last packet. Either way, what is calculated is delay. Of course jitter could result in the delay for the first packet and for the last packet to not be the same. However, this is not a systematic problem.

7/ In Packet Presented NTP timestamp definition in section 6, Clarify that presentation time does not include any intentionally introduced delay. Discuss presentation estimates. Suggested replacement for the first sentence of the definition: "This timestamp reflects the wall clock time at the earliest moment the data contained in the first octet of the associated RTP packet could be presented to the user if no IDMS-directed delay were introduced. In some cases, due to additional delays in external equipment or sound propagation time through air, actual presentation time to the user may not be accurately knowable. For a comfortable media experience, it is preferable to delay audio with respect to video than vise-versa. Pesimistic (longer) assumptions about playout delay should be used for video and optimistic (shorter) ones for audio."

I agree. I completely forgot our previous discussion on this topic. I will include this sentence in the next version.

8/ Clarify use of 0 value in Packet Presented NTP timestamp definition in section 6. If _no presentation timestamp is available_ then is shall be set to 0.

Ok.

9/  Packet Presented NTP timestamp in section 7. "This timestamp reflects the wall clock time at the reference client at the moment the data contained in the first octet of the associated RTP packet _should be presented_ to the user. Change to directive language.

Ok.

10/ SyncGroupID definition in sections 9 and 10. Remove "in network byte order" qualifier. This qualifier may cause confusion, is not necessary and is not applied to attribute definitions in RFC 4566.

Ok.

11/ Editorial correction and clarification in section 13: "Normally, the processing and play-out of the same media stream by _similar_ receivers will take roughly the same amount of time. By synchronizing on packet arrival times, you may _lose_ some accuracy, but it will be adequate for many applications,

Ok.

Kevin Gross
+1-303-447-0517
Media Network Consultant
AVA Networks - www.AVAnw.com<http://www.avanw.com/>, www.X192.org<http://www.X192.org>

This e-mail and its contents are subject to the DISCLAIMER at http://www.tno.nl/emaildisclaimer

---

From: Kevin Gross <[email-masked]>
Date: Thu, 15 Dec 2011 12:08:07 -0700

I believe what we want to measure is the interval between
the origination time for the first octet of packets at the sender and the
time the receiver has usable data. I believe the safe assumption is that
the receiver does not have usable data for a particular RTP timestamp until
he is sure he has received the last octet of the last packet associated
with that timestamp.

We can leave things as currently proposed, timestamp the first octet of the
first packet and assume that with all other receivers timestamping in the
same manner, synchronization based on packet receipt timestamps will work
out.

Or we can suggest a way to timestamp the last octet of the last packet.
Timestamping the last octet as oppose to the first is actually easier
because that's where the network interface will typically generate an
interrupt. Identifying the last packet in the presence of reordering and
packet loss is a bit tricky. I can make a more fully-formed proposal, but
before going to the trouble I'd like to get agreement on what we want to be
measuring here.

Kevin Gross

On Thu, Dec 15, 2011 at 1:56 AM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

> I’m not sure whether there is a functional difference between indicating
> the arrival of the first packet or the last packet. Either way, what is
> calculated is delay. Of course jitter could result in the delay for the
> first packet and for the last packet to not be the same. However, this is
> not a systematic problem.
>
> [F&M] You are right.

[F&M] We are not sure what you mean here. We should discuss this issue but
we think we should define it as the last received RTP packet among all the
sent RTP packets with the same RTP timestamp (regardless of its sequence
number).

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Mon, 19 Dec 2011 09:16:22 +0000

Hi Kevin,

I think the question here is whether there is reason for there to be a systematic difference in transport delay between the first octet and the last octet (i.e. is there a reason for the last octet of an RTP packet to always take longer than the first octet?).

Of course jitter might cause some octets to experience a slightly longer or shorter delay than others, but I don't believe there is a reason to assume this is systematic. If this is the case, then I don't see a reason to further complicate things by measuring the packet arrival of the last octet instead of the first.

It is important to note that IDMS will always be, by its very nature, slightly behind the facts. Apart from the fact that it would not be very useful/scalable, RTCP rules out reporting on every single RTP packet. This means that IDMS will only use measure the delay of some subset of all RTP packets. It therefore don't think it matters whether this measurement happens on the first or the last octet, as long as every receiver does it in the same way.

Ray

From: [email-masked] [mailto:[email-masked]] On Behalf Of Kevin Gross
Sent: donderdag 15 december 2011 20:08
To: [email-masked]
Subject: Re: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02

I believe what we want to measure is the interval between the origination time for the first octet of packets at the sender and the time the receiver has usable data. I believe the safe assumption is that the receiver does not have usable data for a particular RTP timestamp until he is sure he has received the last octet of the last packet associated with that timestamp.

We can leave things as currently proposed, timestamp the first octet of the first packet and assume that with all other receivers timestamping in the same manner, synchronization based on packet receipt timestamps will work out.

Or we can suggest a way to timestamp the last octet of the last packet. Timestamping the last octet as oppose to the first is actually easier because that's where the network interface will typically generate an interrupt. Identifying the last packet in the presence of reordering and packet loss is a bit tricky. I can make a more fully-formed proposal, but before going to the trouble I'd like to get agreement on what we want to be measuring here.

Kevin Gross

On Thu, Dec 15, 2011 at 1:56 AM, Brandenburg, R. (Ray) van <[email-masked]<mailto:[email-masked]>> wrote:

[F&M] You are right.

[F&M] We are not sure what you mean here. We should discuss this issue but we think we should define it as the last received RTP packet among all the sent RTP packets with the same RTP timestamp (regardless of its sequence number).
This e-mail and its contents are subject to the DISCLAIMER at http://www.tno.nl/emaildisclaimer

---

From: Kevin Gross <[email-masked]>
Date: Mon, 19 Dec 2011 21:55:27 -0700

Scenario: Two devices playing video. Both on the same subnet. One connected
via gigabit Ethernet, the other via 802.11b. If we timestamp to the
beginning of the first packet, they both report the same delivery time.
IDMS will see no need to introduce any compensating delay. But, due to
the slower link, the 802.11b device does not receive all data for the frame
until potentially almost a frame later. I would expect presentation to be a
frame delayed with respect to the gigabit device which receives all data
for the frame within a few microseconds.

I don't know if that's a serious enough discrepancy to fret over - if you
care about one frame, you probably want to be syncing to presentation time
not receipt time. But, the other snag with referencing to first byte is
that network interface hardware naturally alerts the network stack to
receipt at last byte. That's the natural place to do a software timestamp.
Of course, it is possible to compute first byte timestamp knowing
interface speed and packet size. But why impose this burden if last byte is
what we want to be timestamping?

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Mon, Dec 19, 2011 at 2:16 AM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

>  Hi Kevin,****
>
> ** **
>
> I think the question here is whether there is reason for there to be a
> systematic difference in transport delay between the first octet and the
> last octet (i.e. is there a reason for the last octet of an RTP packet to
> always take longer than the first octet?).****
>
> ** **
>
> Of course jitter might cause some octets to experience a slightly longer
> or shorter delay than others, but I don’t believe there is a reason to
> assume this is systematic. If this is the case, then I don’t see a reason
> to further complicate things by measuring the packet arrival of the last
> octet instead of the first.****
>
> ** **
>
> It is important to note that IDMS will always be, by its very nature,
> slightly behind the facts. Apart from the fact that it would not be very
> useful/scalable, RTCP rules out reporting on every single RTP packet. This
> means that IDMS will only use measure the delay of some subset of all RTP
> packets. It therefore don’t think it matters whether this measurement
> happens on the first or the last octet, as long as every receiver does it
> in the same way. ****
>
> ** **
>
> Ray****
>
> ** **
>
> ** **
>
> *From:* [email-masked] [mailto:[email-masked]] *On Behalf Of *Kevin
> Gross
> *Sent:* donderdag 15 december 2011 20:08
> *To:* [email-masked]
> *Subject:* Re: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02**
> **
>
> ** **
>
> with that timestamp.****
> ** **
> We can leave things as currently proposed, timestamp the first octet of
> the first packet and assume that with all other receivers timestamping in
> the same manner, synchronization based on packet receipt timestamps will
> work out.****
>
> ** **
>
> measuring here.****
> Kevin Gross****
> ** **
> [email-masked]> wrote:****
>
>  ****
>
> generation and reordering by the network.****
>
>  ****
>
> not a systematic problem. ****
>
> ** **
>
> [F&M] You are right.****
>
>
> number). ****

---

From: "Brandenburg, R. (Ray) van" <[email-masked]>
Date: Wed, 21 Dec 2011 10:09:27 +0000

Hi Kevin,

> Scenario: Two devices playing video. Both on the same subnet. One connected via gigabit Ethernet, the other via 802.11b. If we timestamp to the beginning of the first packet, they both report the same delivery time. IDMS will see no need to introduce any compensating delay. But, due to the slower link, the 802.11b device does not receive all data for the frame until potentially almost a frame later. I would expect presentation to be a frame delayed with respect to the gigabit device which receives all data for the frame within a few microseconds.

I don't see why both devices would report the same delivery time? If the 802.11b link is slower, than it would receive the packet at a later instant and would report a later timestamp right? Or am I missing something here?

> But, the other snag with referencing to first byte is that network interface hardware naturally alerts the network stack to receipt at last byte. That's the natural place to do a software timestamp. Of course, it is possible to compute first byte timestamp knowing interface speed and packet size. But why impose this burden if last byte is what we want to be timestamping?

I think we might be talking about two different things here. The term 'first octet' refers to the RTP layer and not the network/transport (IP/UDP/TCP) layer. It is only meaningful in situations where the size of for example a video frame (e.g. an I-frame) is so big that it is fragmented over different RTP packets having the same RTP timestamp but different RTP sequence numbers. The network interface will only pass the packet to the RTP layer once the entire UDP/TCP packet (including the last byte) has been received. However, from an RTP point of view, this might not mean that the entire RTP packet has been received - that is, more RTP packets with the same RTP timestamp but with a different sequence number, might still need to arrive.

Ray

From: Kevin Gross [mailto:[email-masked]]
Sent: dinsdag 20 december 2011 5:55
To: Brandenburg, R. (Ray) van
Cc: [email-masked]
Subject: Re: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02

Scenario: Two devices playing video. Both on the same subnet. One connected via gigabit Ethernet, the other via 802.11b. If we timestamp to the beginning of the first packet, they both report the same delivery time. IDMS will see no need to introduce any compensating delay. But, due to the slower link, the 802.11b device does not receive all data for the frame until potentially almost a frame later. I would expect presentation to be a frame delayed with respect to the gigabit device which receives all data for the frame within a few microseconds.

I don't know if that's a serious enough discrepancy to fret over - if you care about one frame, you probably want to be syncing to presentation time not receipt time. But, the other snag with referencing to first byte is that network interface hardware naturally alerts the network stack to receipt at last byte. That's the natural place to do a software timestamp. Of course, it is possible to compute first byte timestamp knowing interface speed and packet size. But why impose this burden if last byte is what we want to be timestamping?

Kevin Gross
+1-303-447-0517
Media Network Consultant

On Mon, Dec 19, 2011 at 2:16 AM, Brandenburg, R. (Ray) van <[email-masked]<mailto:[email-masked]>> wrote:
Hi Kevin,

Ray

From: [email-masked]<mailto:[email-masked]> [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Kevin Gross
Sent: donderdag 15 december 2011 20:08
To: [email-masked]<mailto:[email-masked]>
Subject: Re: [AVTCORE] Additional comments: draft-ietf-avtcore-idms-02

Kevin Gross

[F&M] You are right.

---

From: Kevin Gross <[email-masked]>
Date: Wed, 21 Dec 2011 09:32:38 -0700

On Wed, Dec 21, 2011 at 3:09 AM, Brandenburg, R. (Ray) van <
[email-masked]> wrote:

>  Hi Kevin,****
>
> ** **
>
> > Scenario: Two devices playing video. Both on the same subnet. One
> connected via gigabit Ethernet, the other via 802.11b. If we timestamp to
> the beginning of the first packet, they both report the same delivery time.
> for the frame within a few microseconds.****
>
> ** **
>
> I don’t see why both devices would report the same delivery time? If the
> 802.11b link is slower, than it would receive the packet at a later instant
> and would report a later timestamp right? Or am I missing something here?*
> ***
>
> **
>

[kg] The beginning of the first packet arrives at both at approximately the
same time. The slower link does not delay start of receipt. Because of the
lower bitrate, it takes longer for the rest of the packet and any
closely-spaced packets following to be received.

>  **
>
> > But, the other snag with referencing to first byte is that network
> interface hardware naturally alerts the network stack to receipt at last
> byte. That's the natural place to do a software timestamp. Of course, it is
> possible to compute first byte timestamp knowing interface speed and packet
> size. But why impose this burden if last byte is what we want to
> be timestamping?****
>
> ** **
>
> I think we might be talking about two different things here. The term
> ‘first octet’ refers to the RTP layer and not the network/transport
> (IP/UDP/TCP) layer. It is only meaningful in situations where the size of
> for example a video frame (e.g. an I-frame) is so big that it is fragmented
> over different RTP packets having the same RTP timestamp but different RTP
> sequence numbers. The network interface will only pass the packet to the
> RTP layer once the entire UDP/TCP packet (including the last byte) has been
> received. However, from an RTP point of view, this might not mean that the
> entire RTP packet has been received – that is, more RTP packets with the
> same RTP timestamp but with a different sequence number, might still need
> to arrive. ** **
>
> **
>
[kg] Yes, I think we're misunderstanding each other. Let's sort out the
point above and come back to this.