Subject: RE: [AVT] Comments on draft-ietf-avt-rtp-hdrext-01.txt

From: "Desineni, Harikishan" <[email-masked]>
Date: Fri, 21 Jul 2006 16:45:18 -0700

With this definition, the "offset" can never take a -ve value. The
definition of offset field could be changed to 24-bit unsigned integer.

> -----Original Message-----
> From: Ron Frederick [mailto:[email-masked]] 
> Sent: Friday, July 21, 2006 1:19 PM
> To: Magnus Westerlund
> Cc: AVT WG
> Subject: Re: [AVT] Comments on draft-ietf-avt-rtp-hdrext-01.txt
> 
> Magnus Westerlund wrote:
> > Ron, I think we have slightly different views on how this 
> is handled. 
> > However I do agree that it makes sense to define the offset in the 
> > following way.
> > 
> > offset = convert_to_RTP_TS(NTP_transmission_instant - 
> > NTP_sampling_instant)
> > 
> > Where NTP_sampling_instant will be the NTP value used in 
> RTCP if the 
> > RTCP packet would contain the same RTP TS value. That will work 
> > independently of if the content is actually sampled or have a 
> > generated time line.
> 
> This computation looks good to me, Magnus, and would produce 
> what I consider to be an "absolute offset" in our previous discussion.
> 
> In practice, a sender may be using its local clock for the 
> NTP_transmission_instant measurement, but I like the 
> comparison you made here about how to convert that to an RTP 
> timestamp. The sender has exactly the same issue when 
> generating RTCP sender or receiver reports. 
> There, they know the transmission time of the RTCP packet on 
> their local clock and then need to convert that to both an 
> NTP timestamp and an equivalent RTP timestamp. However they 
> do this conversion from local clock to RTP timestamp for RTCP 
> should also be used when computing the toffset value.
> --
> Ron Frederick
> [email-masked]
> 
> _______________________________________________
> Audio/Video Transport Working Group
> [email-masked]
> https://www1.ietf.org/mailman/listinfo/avt
> 

_______________________________________________
Audio/Video Transport Working Group
[email-masked]
https://www1.ietf.org/mailman/listinfo/avt

---

From: Magnus Westerlund <[email-masked]>
Date: Mon, 24 Jul 2006 11:35:39 +0200

Yes,

Actually it needs to be a signed value but in RTP TS Ticks. Sorry for 
not being cleared. It needs to maintain the sign and allow for cases 
when the transmission time is previous to "sampling instant" as this may 
occur in streaming.

Cheers

Magnus

-- 

Magnus Westerlund

Multimedia Technologies, Ericsson Research EAB/TVA/A
----------------------------------------------------------------------
Ericsson AB                | Phone +46 8 4048287
Torshamsgatan 23           | Fax   +46 8 7575550
S-164 80 Stockholm, Sweden | mailto: [email-masked]

---

From: Ron Frederick <[email-masked]>
Date: Mon, 24 Jul 2006 09:26:46 -0700

> when the transmission time is previous to "sampling instant" as this may 

I'm guessing here that you are thinking of stored content. Please 
correct me if you have something else in mind.

For content being captured live, clearly one cannot send data before one 
captures it. So, in this case I don't see how the offset could ever be 
negative.

However, for stored (video on demand) content played from a storage 
server, the concept of "sampling instant" becomes somewhat arbitrary, so 
I could imagine designs in which you might choose a sampling instant 
which made this possible. For instance, because of the reordering of I, 
P, and B frames in MPEG, a streaming server would need to transmit a 
sequence such as:

	I0 B0 B1 B2 P0 B3 B4 B5 P1 B6 B7 B8 P2 ...

as:

	I0 P0 B0 B1 B2 P1 B3 B4 B5 P2 B6 B7 B8 ...

The simplest thing to do would be to treat the sampling instant of the 
first frame as being equal to its send time and then pace all the 
packets out after that as smoothly as possible, but then you might end 
up sending P0's data well before its apparent sampling time, since you 
need to get both P0's data and B0's data to the receiver in time for B0 
to be displayed. For a live stream, there would be an encoding delay 
here that would push all of these send times forward compared to the 
sampling time to make all of the offsets positive, as you couldn't start 
encoding B0 until after P0 had been captured, but a storage server might 
not have this information. Is that the case you are thinking of where 
the offsets come out negative?

Ideally, I'd prefer a design where a video-on-demand stream was forced 
to look exactly the same as a live stream to make it easier on 
receivers. However, we also need to consider what information a sender 
has available, and a storage server may not have enough information in 
the stored file to reconstruct a true picture of the original sampling 
instant and encoding delays. So, unless we require the sender to scan 
the entire file (which is impractical), we may need to accept the fact 
that it will have to make an arbitrary guess about the sampling instant 
that could cause some offsets to end up being negative.
-- 
Ron Frederick
[email-masked]

---

From: Magnus Westerlund <[email-masked]>
Date: Tue, 25 Jul 2006 10:00:38 +0200

Ron Frederick wrote:
>> when the transmission time is previous to "sampling instant" as this 
>> may occur in streaming.
> 

Actually although most commonly used with stored content, much of the 
mechanism are applicable if you can allow a few seconds of delay at the 
encoding and transmission stages.

> the offsets come out negative?

Yes, that is one case. This also occurs when you start do advanced 
tricks for reliability. One thing that most people has discussed is 
sending the I frames far earlier than the P frames that are sent in 
their turn somewhat earlier than the B frames that are delivered just in 
time. Such a scheme for example allow for selective retransmission to 
make the most for important data.

> that could cause some offsets to end up being negative.

Well this is much up to the application specification. In 3GPP 
packet-based streaming (PSS TS 26.234 rel6) there is already the 
definitions of the buffers and procedures that allows one to send a 
stream with some packets being sent very early, and some just in time 
for consumption. In other applications like IP based cable TV there is 
other constraints that changes what you can do.

In some sense if you like a stored stream to look like a live stream, 
the only thing you need is to introduce sufficient delay to send all 
packets with 0 or positive offset. However how big that offset is that 
depends very much on the encoding and how much delay one has allowed in 
that process.

The general problem is that you like a definition that are applicable in 
all the known applications and which allows them to work.

Cheers

Magnus Westerlund

---

From: Ron Frederick <[email-masked]>
Date: Fri, 28 Jul 2006 13:39:57 -0700

>> correct me if you have something else in mind.

Yes, the mechanisms are applicable, but in this case the offset would 
still reflect the actual capture time, so all the offsets would be positive.

>> For content being captured live, clearly one cannot send data before 
>> one captures it. So, in this case I don't see how the offset could 
>> ever be negative.
>>
>> server, the concept of "sampling instant" becomes somewhat arbitrary, 
>> so I could imagine designs in which you might choose a sampling 
>> instant which made this possible. For instance, because of the 
>> reordering of I, P, and B frames in MPEG, a streaming server would 
>> need to transmit a sequence such as:
>>
>> need to get both P0's data and B0's data to the receiver in time for 
>> B0 to be displayed. For a live stream, there would be an encoding 
>> delay here that would push all of these send times forward compared to 
>> the sampling time to make all of the offsets positive, as you couldn't 
>> start encoding B0 until after P0 had been captured, but a storage 
>> server might not have this information. Is that the case you are 
>> thinking of where the offsets come out negative?
> 

I never really considered what would happen with retransmissions in this 
case, but that makes sense.

>> that it will have to make an arbitrary guess about the sampling 
>> instant that could cause some offsets to end up being negative.
> 

The delay would be there regardless of whether the "offset" field in the 
extension reflected it or not. In order to play packets smoothly, the 
receiver will need to buffer enough to cope with the worst case delay. 
Packets that claim to have a negative offset will just be buffered for 
that much longer, but the overall buffering time of the "most negative" 
packet will be the same either way.

Setting an "apparent capture time" far enough in the past to make all 
the offsets positive would be nice if it were possible, but my concern 
is that a streaming server which is playing back a stored file might not 
know what the worst-case delay is. This offset must be chosen before the 
first packet is sent. If it chooses too small a value and later decides 
it wants to send some packet very early, it will end up with a negative 
offset.

I suppose all of this is already determined, though, if you go with the 
model that the streaming server is already computing some mapping 
between its RTP timestamps and NTP time in RTCP. The offset it reports 
would be the difference between that NTP time and the NTP time of when 
it actually sent the packet, converted back to RTP timestamp units. So, 
the question of what the "apparent capture time" is was already 
determined when the streaming server picked what NTP timestamp to use in 
RTCP. Has there been any guidance in any of the specs for how a 
streaming server should choose this value when sending stored content?
-- 
Ron Frederick
[email-masked]