Subject: [AVTCORE] Question on multi-party RTT handling (draft-hellstrom-avtcore-multi-party-rtt-source-03)

From: Yong Xin <[email-masked]>
Date: Wed, 20 May 2020 22:28:35 +0000

Dear Mr. Hellstrom,

I have a question about how to use RTT mixer (rtt-mix) method with "text/red" format for multi-party call handling, as defined in your IETF draft https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-source-03.
  4. Use of fields in the RTP packets

   RFC 4103<https://tools.ietf.org/html/rfc4103>[RFC4103<https://tools.ietf.org/html/rfc4103>] specifies use of RFC 3550<https://tools.ietf.org/html/rfc3550> RTP[RFC3550], and a

   redundancy format "text/red" for increased robustness.  This

   specification updates RFC 4102<https://tools.ietf.org/html/rfc4102>[RFC4102<https://tools.ietf.org/html/rfc4102>] and RFC 4103<https://tools.ietf.org/html/rfc4103>[RFC4103<https://tools.ietf.org/html/rfc4103>] by

   introducing a rule for populating and using the CSRC-list in the RTP

   packet in order to enhance the performance in multi-party RTT

   sessions.

   When transmitted from a mixer, the first member in the CSRC-list

   SHALL contain the SSRC of the source of the primary T140block in the

   packet.  The second and further members in the CSRC-list SHALL

   contain the SSRC of the source of the first, second, etc redundant

   generations of T140blocks included in the packet. ( the recommended

   level of redundancy is to use one primary and two redundant

   generations of T140blocks.)  In some cases, a primary or redundant

   T140block is empty, but is still represented by a member in the

   redundancy header.  For such cases, the corresponding CSRC-list

   member MUST also be included.

   The CC field SHALL show the number of members in the CSRC list.

   Note: This specification departs from section 4 of RFC 2198<https://tools.ietf.org/html/rfc2198#section-4> [RFC2198<https://tools.ietf.org/html/rfc2198>]

   which associates the whole of the CSRC-list with the primary data and

   assumes that the same list applies to reconstructed redundant data.

   In the present specification a T140block is associated with exactly

   one CSRC list member as described above.  Also RFC 2198<https://tools.ietf.org/html/rfc2198> [RFC2198<https://tools.ietf.org/html/rfc2198>]

   anticipates infrequent change to CSRCs; implementers should be aware

   that the order of the CSRC-list according to this specification will

   vary during transitions between transmission from the mixer of text

   originated by different participants.

   The picture below shows a typical RTP packet with multi-party RTT

   contents and coding according to the present specification.

       0                   1                   2                   3
       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |V=2|P|X| CC=3  |M|  "RED" PT   |   sequence number of primary  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |               timestamp of primary encoding "P"               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |           synchronization source (SSRC) identifier            |
      +=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+=+
      |  CSRC list member 1 = SSRC of source of "P"                   |
      |  CSRC list member 2 = SSRC of source of "R1"                  |
      |  CSRC list member 3 = SSRC of source of "R2"                  |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |1|   T140 PT   |  timestamp offset of "R2" | "R2" block length |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |1|   T140 PT   |  timestamp offset of "R1" | "R1" block length |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |0|   T140 PT   | "R2" T.140 encoded redundant data             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+---------------+
      |   |  "R1" T.140 encoded redundant data        |               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+         +-+-+-+
      |              "P" T.140 encoded primary data             |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

       Figure 1: text/red packet with sources indicated in the CSRC-list.

At every transmission time, the mixer can use the primary data block to send new texts from one source, but new texts from other sources will have to wait in their queue for their turn. I assume it is a round-robin fashion to determine the next source. The default text transmission interval is 300ms, which means the texts from other sources have to wait in the queue for at least 300ms before they can be transmitted. I can see you have recommended to reduce the transmission interval from 300ms to 100ms to reduce this delay, but in the case of large conference and assuming every participant is typing the text simultaneously, the waiting time in the queue will become longer. For example, in a 10-party conference, even with 100ms transmission interval, the new texts from last participant will wait for 9x100ms = 900ms to send. This delay will be too long for some emergency service. Increasing the redundancy level will only help to recovery from more consecutive packet loss, but it does not help to reduce this delay. So it looks to me this method is not ideal for large conference, is my understanding correct? Has this issue been discussed in the IETF meeting before? Do you have any recommendation to solve this problem?

Thanks,
Yong

---

From: Gunnar Hellström <[email-masked]>
Date: Thu, 21 May 2020 09:39:14 +0200

Dear  Yong,

Thanks for a good question.

The draft you are asking about has been replaced by this one:

https://datatracker.ietf.org/doc/draft-ietf-avtcore-multi-party-rtt-mix/

and it is modified at the point of your question, and partly because of 
the issue you saw with the draft you looked in. More follows inline,

Den 2020-05-21 kl. 00:28, skrev Yong Xin:
>
> Dear Mr. Hellstrom,
>
> I have a question about how to use RTT mixer (rtt-mix) method with 
> “text/red” format for multi-party call handling, as defined in your 
> IETF draft 
> https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-source-03. 
>
>
>
> RFC 4103 <https://tools.ietf.org/html/rfc4103>[RFC4103 
> <https://tools.ietf.org/html/rfc4103>] specifies use of RFC 3550 
> <https://tools.ietf.org/html/rfc3550> RTP[RFC3550], and a
>    specification updates RFC 4102 
> <https://tools.ietf.org/html/rfc4102>[RFC4102 
> <https://tools.ietf.org/html/rfc4102>] and RFC 4103 
> <https://tools.ietf.org/html/rfc4103>[RFC4103 
> <https://tools.ietf.org/html/rfc4103>] by
>    Note: This specification departs from section 4 of RFC 2198 
> <https://tools.ietf.org/html/rfc2198#section-4> [RFC2198 
> <https://tools.ietf.org/html/rfc2198>]
>    one CSRC list member as described above.  Also RFC 2198 
> <https://tools.ietf.org/html/rfc2198> [RFC2198 
> <https://tools.ietf.org/html/rfc2198>]
> At every transmission time, the mixer can use the primary data block 
> to send new texts from one source, but new texts from other sources 
> will have to wait in their queue for their turn. I assume it is a 
> round-robin fashion to determine the next source. The default text 
> transmission interval is 300ms, which means the texts from other 
> sources have to wait in the queue for at least 300ms before they can 
> be transmitted. I can see you have recommended to reduce the 
> transmission interval from 300ms to 100ms to reduce this delay, but in 
> the case of large conference and assuming every participant is typing 
> the text simultaneously, the waiting time in the queue will become 
> longer. For example, in a 10-party conference, even with 100ms 
> transmission interval, the new texts from last participant will wait 
> for 9x100ms = 900ms to send. This delay will be too long for some 
> emergency service. Increasing the redundancy level will only help to 
> recovery from more consecutive packet loss, but it does not help to 
> reduce this delay. So it looks to me this method is not ideal for 
> large conference, is my understanding correct? Has this issue been 
> discussed in the IETF meeting before? Do you have any recommendation 
> to solve this problem?
>
Your understanding is correct. There is discussion about various ways to 
arrange the mixing in another draft: 
https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions 
<https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00>

It is slightly outdated now, but it contains reasoning about performance 
and other aspects of different solutions.

The current draft replacing the one you read, specifies a packet format 
that enables new text from up to 16 simultaneous text sources. It is 
possible to send text from more simultaneous sending users, but then 
there will be a short delay for some. The delay for 1-16 simultaneous 
texters will vary between 0 and 300 milliseconds.

Even in a large conference, it will in most cases be only one 
participant sending real-time text, but occasionally two or three. It 
will be as for voice or for sign language in video: It will be 
unmanageable for the participants to perceive media from many sources 
simultaneously. I agree that for text, the opportunities are a bit 
better than for audio and video. The text at least stays and is readable 
in a well arranged display where the participants can catch up reading 
if there were many sending simultaneously.

You mention the emergency call with 10 participants as an example where 
a delay of 900 ms would be a risk. In the type of emergency call I think 
of, where one person in an emergency calls the emergency number and get 
a connection with an emergency call taker, I can only imagine there be 
in very unusual cases up to maybe 5 participants, most often taking 
turns nicely in sending text.

It can for example be the user, the call taker, a language translator, a 
first responder and an expert in chemical danger. The simultaneous 
typing that may occur will be e.g. the user coming with more information 
while the first responder types some instructions for how to handle the 
case. The others will in most cases wait for their turn.

The more common emergency call will have three participants: The calling 
user, the call taker and a first responder or other agent. And then the 
two people in the service know how to take turns. So it will be a 
maximum of two participants typing simultaneously.

I can imagine a completely other kind of emergency conference, where 
people call in and report accidents they have seen to check if they are 
already handled, and they get reports about ongoing emergencies. If it 
is at all realistic to set up such service as a conference call, there 
would indeed be small delays before some text is presented. However, the 
900 ms in your example is the time that a person normally types a word, 
and the person supposed to act on all these text streams may need to 
switch from reading another source to the end and then move to respond 
or look at the new source. That will always take more than one second. 
So even here, the replaced draft would result in good performance. And 
this is not what is meant with an emergency call.

Maybe there will be some other applications with unmanaged conferences 
with real-time text where a lot of simultaneous typing will occur.  
Therefore I moved to specifying for up to 16 simultaneous sources.

There are also both human and regulatory requirements saying that 
real-time text MUST not be delayed more than 500 ms or 1 second 
(depending on what document you read, and where the delay is measured.) 
So that should be obeyed for normal cases.

In the replaced draft you refer to, the format is called "text/red" just 
as for RFC 4103, and negotiated by an sdp attribute. I got indications 
off-list that that would not be allowed. The change in the use of the 
CSRC list from what is stated in RFC 2198 would be too big. Therefore I 
needed to move to call it a new format "text/rex", and negotiate it by 
payload types in the m-line. When I realized that I needed to take that 
step, it was also natural to improve the format to be able to carry more 
text without introducing delays.

Do you agree that the current draft 
draft-ietf-avtcore-multi-party-rtt-mix-01 solves your concerns?

Thanks,

Gunnar

-- 
Gunnar Hellström
GHAccess
[email-masked]

---

From: Yong Xin <[email-masked]>
Date: Fri, 22 May 2020 00:53:21 +0000

Dear Hellstrom,

Thanks for the quick response. The latest spec does address my concern. I have some follow-up questions:

  *   The new payload format "text/rex" can be used with or without redundancy. When redundancy is used, mixer has to use the same redundancy level when transmitting texts from multiple sources. If the different party in the same conference has negotiated a different redundancy level, the mixer has to pick the lowest level to use, right? But in case there's one party has negotiated "text/rex" without any redundancy level, does that mean mixer has to turn of the redundancy for this conference? Does mixer need to change the redundancy level up and down dynamically as user joins or leaves the conference? Does mixer need to send re-INVITE to re-negotiate the redundancy level with other party when such change happens?

  *   In section 12, I noticed the 150cps recommendation is still there and has been made as default value for the new packet format, but the transmission interval is back to 300ms (the recommended interval was 100ms in the old spec). I guess with the new packet format, it is not required to use the shorter transmission interval any more. And you mentioned these characteristics provide for smooth flow of text with acceptable latency from at least 32 sources simultaneously. Since the new packet format can support up to 16 sources per packet, the text from 32 sources will have be transmitted in turn. If my calculation is correct, with 300ms transmission interval and redundancy level 2, it will take 900ms (one primary + 2 redundant) for mixer to switch from first 16 sources to next 16 sources, so the delay is about 900ms. Is this the acceptable latency in your mind?

  *   There're quite a few updates in the spec in the last couple of months. When do you expect this IETF draft will get finalized and approved?

Regards,
Yong

From: Gunnar Hellström <[email-masked]>
Sent: Thursday, May 21, 2020 12:39 AM
To: Yong Xin <[email-masked]>; [email-masked]
Subject: Re: Question on multi-party RTT handling (draft-hellstrom-avtcore-multi-party-rtt-source-03)

The e-mail below is from an external source. Please do not open attachments or click links from an unknown or suspicious origin.

Dear  Yong,

and it is modified at the point of your question, and partly because of the issue you saw with the draft you looked in. More follows inline,
Den 2020-05-21 kl. 00:28, skrev Yong Xin:
Dear Mr. Hellstrom,

   sessions.

       0                   1                   2                   3

Your understanding is correct. There is discussion about various ways to arrange the mixing in another draft: https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions<https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00>

It is slightly outdated now, but it contains reasoning about performance and other aspects of different solutions.

The current draft replacing the one you read, specifies a packet format that enables new text from up to 16 simultaneous text sources. It is possible to send text from more simultaneous sending users, but then there will be a short delay for some. The delay for 1-16 simultaneous texters will vary between 0 and 300 milliseconds.

Even in a large conference, it will in most cases be only one participant sending real-time text, but occasionally two or three. It will be as for voice or for sign language in video: It will be unmanageable for the participants to perceive media from many sources simultaneously. I agree that for text, the opportunities are a bit better than for audio and video. The text at least stays and is readable in a well arranged display where the participants can catch up reading if there were many sending simultaneously.

You mention the emergency call with 10 participants as an example where a delay of 900 ms would be a risk. In the type of emergency call I think of, where one person in an emergency calls the emergency number and get a connection with an emergency call taker, I can only imagine there be in very unusual cases up to maybe 5 participants, most often taking turns nicely in sending text.

It can for example be the user, the call taker, a language translator, a first responder and an expert in chemical danger. The simultaneous typing that may occur will be e.g. the user coming with more information while the first responder types some instructions for how to handle the case. The others will in most cases wait for their turn.

The more common emergency call will have three participants: The calling user, the call taker and a first responder or other agent. And then the two people in the service know how to take turns. So it will be a maximum of two participants typing simultaneously.

I can imagine a completely other kind of emergency conference, where people call in and report accidents they have seen to check if they are already handled, and they get reports about ongoing emergencies. If it is at all realistic to set up such service as a conference call, there would indeed be small delays before some text is presented. However, the 900 ms in your example is the time that a person normally types a word, and the person supposed to act on all these text streams may need to switch from reading another source to the end and then move to respond or look at the new source. That will always take more than one second. So even here, the replaced draft would result in good performance. And this is not what is meant with an emergency call.

Maybe there will be some other applications with unmanaged conferences with real-time text where a lot of simultaneous typing will occur.  Therefore I moved to specifying for up to 16 simultaneous sources.

There are also both human and regulatory requirements saying that real-time text MUST not be delayed more than 500 ms or 1 second (depending on what document you read, and where the delay is measured.) So that should be obeyed for normal cases.

In the replaced draft you refer to, the format is called "text/red" just as for RFC 4103, and negotiated by an sdp attribute. I got indications off-list that that would not be allowed. The change in the use of the CSRC list from what is stated in RFC 2198 would be too big. Therefore I needed to move to call it a new format "text/rex", and negotiate it by payload types in the m-line. When I realized that I needed to take that step, it was also natural to improve the format to be able to carry more text without introducing delays.

Do you agree that the current draft draft-ietf-avtcore-multi-party-rtt-mix<https://datatracker.ietf.org/doc/draft-ietf-avtcore-multi-party-rtt-mix/>-01 solves your concerns?

Thanks,

Gunnar

Thanks,
Yong

--

Gunnar Hellström

GHAccess

[email-masked]<mailto:[email-masked]>

---

From: Gunnar Hellström <[email-masked]>
Date: Fri, 22 May 2020 11:40:40 +0200

Hi Yong,

Thanks for good questions,

Den 2020-05-22 kl. 02:53, skrev Yong Xin:
>
> Dear Hellstrom,
>
> Thanks for the quick response. The latest spec does address my 
> concern. I have some follow-up questions:
>
>   * The new payload format “text/rex” can be used with or without
>     redundancy. When redundancy is used, mixer has to use the same
>     redundancy level when transmitting texts from multiple sources. If
>     the different party in the same conference has negotiated a
>     different redundancy level, the mixer has to pick the lowest level
>     to use, right?
>
No. There are two sides of this answer:

The mixer should do separate mixing for each recipient, using the 
redundancy level agreed with each recipient. This is also because the 
users do not want to see their own transmitted text being received from 
the mixer. The own text is displayed locally by the endpoints. If the 
recipient does not support "text/rex", the mixer also need to do the 
mixing for multi-party unaware endpoints using the "text/red" format  
described in section 13.2.

And the mixer must recover from loss in reception from each source and 
create a queue of clean text from each source before composing the 
packets for transmission. The mixer cannot just resend received packet 
contents with redundancy, because the recovery mechanism requires the 
sequence number gaps for loss detection, and the mixer must create its 
own sequence number series in the transmission.

>   * But in case there’s one party has negotiated “text/rex” without
>     any redundancy level, does that mean mixer has to turn of the
>     redundancy for this conference? Does mixer need to change the
>     redundancy level up and down dynamically as user joins or leaves
>     the conference? Does mixer need to send re-INVITE to re-negotiate
>     the redundancy level with other party when such change happens?
>
 From the logic above, the answers on these questions are: no. I realize 
that an explanation should be inserted in the beginning of section 3. 
"Actions at transmission by the mixer" to clarify that the source for 
transmission from the mixer is clean text in separate queues regardless 
of which format or protocol they used in the individual receptions.
>
>   * In section 12, I noticed the 150cps recommendation is still there
>     and has been made as default value for the new packet format, but
>     the transmission interval is back to 300ms (the recommended
>     interval was 100ms in the old spec). I guess with the new packet
>     format, it is not required to use the shorter transmission
>     interval any more.
>
The transmission interval is mentioned in two paragraphs in section 3. 
One saying that the default is 300 ms, the other saying:

"For multi-party operation, it is RECOMMENDED that the mixer sends a 
packet to each receiver as soon as text has been received from asource 
as long as the maximum number of characters per secondindicated by the 
recipient is not exceeded, and also the number ofpackets sent per second 
to a recipient is kept under a specifiednumber.This number SHALL be 10 
if no other limit is applied for the application.The intention is to 
keep the latency introduced by themixer low."

This is intended to create a balance between low latency and protection 
against bursty packet loss. Even if the latency requirements from 
real-time text users are much lower than from audio and video users, a 
low latency is appreciated, and latency of over 2 seconds end-to-end 
creates conversation problems.  Therefore, this paragraph about when to 
transmit will self-regulate to about 100 ms packet interval from about 3 
simultaneous typing sources.

The 300 ms default assures that the remaining redundancy transmissions 
will be sent even shortly after all sources have stopped typing.

However, this algorithm may make the protection against bursty loss 
weaker than with a steady 300 ms interval. With between 17 and 32 
simultaneous typing users, the latency caused by the mixer will be 
around 300 ms and then passes both regulatory and human requirements.

Now, even if it passes the requirements, 32 is a very unrealistic number 
of simultaneous typing users. In audio conferences it is only possible 
to perceive one source at a time well. The benefit of enabling more is 
just for noticing that someone else want to say something. Requirements 
for this work are collected in 
draft-hellstrom-avtcore-multi-party-rtt-solutions-00, and there the 
performance requirements are set to be valid for up to 5 simultaneously 
transmitting users and the delay caused by the mixer to be less than 500 
ms. I think we should design for these figures.

>   * And you mentioned these characteristics provide for smooth flow of
>     text with acceptable latency from at least 32 sources
>     simultaneously. Since the new packet format can support up to 16
>     sources per packet, the text from 32 sources will have be
>     transmitted in turn. If my calculation is correct, with 300ms
>     transmission interval and redundancy level 2, it will take 900ms
>     (one primary + 2 redundant) for mixer to switch from first 16
>     sources to next 16 sources, so the delay is about 900ms. Is this
>     the acceptable latency in your mind?
>
See discussions above. Maybe regulators need to say how many 
simultaneous users the requirements are for. I think 5 is a high and 
good figure even if the discussion above indicates 32 to be possible.
>
>  *
>
>   * There’re quite a few updates in the spec in the last couple of
>     months. When do you expect this IETF draft will get finalized and
>     approved?
>
Yes, I hope for a short period of discussions like yours that I 
appreciate to sort out the main principles, and then a period of the 
different levels of last calls and refinement editing. Even if one would 
hope for a more rapid progress, a realistic milestone for sending it to 
IESG has been set for February 2021. It would be good if various 
organisations who would reference it in their specifications (as 3GPP, 
ATIS, NENA, ETSI etc.) would take a look already now and assess if it is 
agreeable and on the right way.

Thanks

Gunnar

>  *
>
> Regards,
>
> Yong
>
> *From:* Gunnar Hellström <[email-masked]>
> *Sent:* Thursday, May 21, 2020 12:39 AM
> *To:* Yong Xin <[email-masked]>; [email-masked]
> *Subject:* Re: Question on multi-party RTT handling 
> (draft-hellstrom-avtcore-multi-party-rtt-source-03)
>
>
> 	
>
> The e-mail below is from an external source. Please do not open 
> attachments or click links from an unknown or suspicious origin.
>
> Dear  Yong,
>
> and it is modified at the point of your question, and partly because 
> of the issue you saw with the draft you looked in. More follows inline,
>
>     “text/red” format for multi-party call handling, as defined in
>     your IETF draft
>     At every transmission time, the mixer can use the primary data
>     block to send new texts from one source, but new texts from other
>     sources will have to wait in their queue for their turn. I assume
>     it is a round-robin fashion to determine the next source. The
>     default text transmission interval is 300ms, which means the texts
>     from other sources have to wait in the queue for at least 300ms
>     before they can be transmitted. I can see you have recommended to
>     reduce the transmission interval from 300ms to 100ms to reduce
>     this delay, but in the case of large conference and assuming every
>     participant is typing the text simultaneously, the waiting time in
>     the queue will become longer. For example, in a 10-party
>     conference, even with 100ms transmission interval, the new texts
>     from last participant will wait for 9x100ms = 900ms to send. This
>     delay will be too long for some emergency service. Increasing the
>     redundancy level will only help to recovery from more consecutive
>     packet loss, but it does not help to reduce this delay. So it
>     looks to me this method is not ideal for large conference, is my
>     understanding correct? Has this issue been discussed in the IETF
>     meeting before? Do you have any recommendation to solve this problem?
>
> Your understanding is correct. There is discussion about various ways 
> to arrange the mixing in another draft: 
> It is slightly outdated now, but it contains reasoning about 
> performance and other aspects of different solutions.
>
> The current draft replacing the one you read, specifies a packet 
> format that enables new text from up to 16 simultaneous text sources. 
> It is possible to send text from more simultaneous sending users, but 
> then there will be a short delay for some. The delay for 1-16 
> simultaneous texters will vary between 0 and 300 milliseconds.
>
> better than for audio and video. The text at least stays and is 
> readable in a well arranged display where the participants can catch 
> up reading if there were many sending simultaneously.
>
> You mention the emergency call with 10 participants as an example 
> where a delay of 900 ms would be a risk. In the type of emergency call 
> I think of, where one person in an emergency calls the emergency 
> number and get a connection with an emergency call taker, I can only 
> imagine there be in very unusual cases up to maybe 5 participants, 
> most often taking turns nicely in sending text.
>
> It can for example be the user, the call taker, a language translator, 
> a first responder and an expert in chemical danger. The simultaneous 
> typing that may occur will be e.g. the user coming with more 
> information while the first responder types some instructions for how 
> to handle the case. The others will in most cases wait for their turn.
>
> The more common emergency call will have three participants: The 
> calling user, the call taker and a first responder or other agent. And 
> then the two people in the service know how to take turns. So it will 
> be a maximum of two participants typing simultaneously.
>
> people call in and report accidents they have seen to check if they 
> are already handled, and they get reports about ongoing emergencies. 
> If it is at all realistic to set up such service as a conference call, 
> there would indeed be small delays before some text is presented. 
> However, the 900 ms in your example is the time that a person normally 
> types a word, and the person supposed to act on all these text streams 
> may need to switch from reading another source to the end and then 
> move to respond or look at the new source. That will always take more 
> than one second. So even here, the replaced draft would result in good 
> performance. And this is not what is meant with an emergency call.
>
> (depending on what document you read, and where the delay is 
> measured.) So that should be obeyed for normal cases.
>
> In the replaced draft you refer to, the format is called "text/red" 
> just as for RFC 4103, and negotiated by an sdp attribute. I got 
> indications off-list that that would not be allowed. The change in the 
> use of the CSRC list from what is stated in RFC 2198 would be too big. 
> Therefore I needed to move to call it a new format "text/rex", and 
> negotiate it by payload types in the m-line. When I realized that I 
> needed to take that step, it was also natural to improve the format to 
> be able to carry more text without introducing delays.
>
> draft-ietf-avtcore-multi-party-rtt-mix 
> <https://datatracker.ietf.org/doc/draft-ietf-avtcore-multi-party-rtt-mix/>-01 
> solves your concerns?
>
> Thanks,
>
> Gunnar
>
>     Thanks,
>
>     Yong
>
> -- 
> Gunnar Hellström
> GHAccess
> [email-masked]  <mailto:[email-masked]>

---

From: Yong Xin <[email-masked]>
Date: Fri, 22 May 2020 23:31:30 +0000

Hi Gunnar,

Thanks for the clarification - see my response inline

Regards,
Yong

From: Gunnar Hellström <[email-masked]>
Sent: Friday, May 22, 2020 2:41 AM
To: Yong Xin <[email-masked]>; [email-masked]
Subject: Re: Question on multi-party RTT handling (draft-hellstrom-avtcore-multi-party-rtt-source-03)

Hi Yong,

  *   The new payload format "text/rex" can be used with or without redundancy. When redundancy is used, mixer has to use the same redundancy level when transmitting texts from multiple sources. If the different party in the same conference has negotiated a different redundancy level, the mixer has to pick the lowest level to use, right?

The mixer should do separate mixing for each recipient, using the redundancy level agreed with each recipient. This is also because the users do not want to see their own transmitted text being received from the mixer. The own text is displayed locally by the endpoints. If the recipient does not support "text/rex", the mixer also need to do the mixing for multi-party unaware endpoints using the "text/red" format  described in section 13.2.

[YX] Understood, the text mixer is providing N-1 mixing, similar to audio mixing, so the user never receive their own transmitted text from the mixer.

And the mixer must recover from loss in reception from each source and create a queue of clean text from each source before composing the packets for transmission. The mixer cannot just resend received packet contents with redundancy, because the recovery mechanism requires the sequence number gaps for loss detection, and the mixer must create its own sequence number series in the transmission.

[YX] I agree what you said here. I think I'm little confused when reading the following paragraph in section 3 of the spec. Let me put an example, there's a 3-party conference and all participants (A, B, C) are conference-aware RTT terminals and support text/rex packet format. User A, B, C negotiates different redundancy level 2, 3, 1 respectively. When mixer transmitting text (source from B & C) to user A, what is the number of redundant generations should be used by the mixer in the transmitted packet? Is it 2 or 1?
   The number of redundant generations of T140blocks to include in
   transmitted packets SHALL be deducted from the SDP negotiation.  It
   SHOULD be set to the minimum of the number declared by the receiver
   and the transmitter.  The same number of redundant generations MUST
   be used for all sources in the transmissions.  The number of
   generations sent to a receiver SHALL be the same during the whole
   session unless it is modified by session renegotiation.

  *   But in case there's one party has negotiated "text/rex" without any redundancy level, does that mean mixer has to turn of the redundancy for this conference? Does mixer need to change the redundancy level up and down dynamically as user joins or leaves the conference? Does mixer need to send re-INVITE to re-negotiate the redundancy level with other party when such change happens?
>From the logic above, the answers on these questions are: no. I realize that an explanation should be inserted in the beginning of section 3. "Actions at transmission by the mixer" to clarify that the source for transmission from the mixer is clean text in separate queues regardless of which format or protocol they used in the individual receptions.

[YX] This is related to the above question. Some clarification in the spec would be helpful.

  *   In section 12, I noticed the 150cps recommendation is still there and has been made as default value for the new packet format, but the transmission interval is back to 300ms (the recommended interval was 100ms in the old spec). I guess with the new packet format, it is not required to use the shorter transmission interval any more.
The transmission interval is mentioned in two paragraphs in section 3. One saying that the default is 300 ms, the other saying:

"For multi-party operation, it is RECOMMENDED that the mixer sends a packet to each receiver as soon as text has been received from a source as long as the maximum number of characters per second indicated by the recipient is not exceeded, and also the number of packets sent per second to a recipient is kept under a specified number.  This number SHALL be 10 if no other limit is applied for the application.  The intention is to keep the latency introduced by the mixer low."

This is intended to create a balance between low latency and protection against bursty packet loss. Even if the latency requirements from real-time text users are much lower than from audio and video users, a low latency is appreciated, and latency of over 2 seconds end-to-end creates conversation problems.  Therefore, this paragraph about when to transmit will self-regulate to about 100 ms packet interval from about 3 simultaneous typing sources.

The 300 ms default assures that the remaining redundancy transmissions will be sent even shortly after all sources have stopped typing.

[YX] So are you saying the recommended transmission interval in the new spec is still 100ms, and the 300ms is actually the time that covers 3 transmissions (one primary transmission plus 2 redundancy transmissions, assuming redundancy level 2), is my understanding correct? I guess this is better clarified in the new spec. The old spec is much clear in this definition.

However, this algorithm may make the protection against bursty loss weaker than with a steady 300 ms interval. With between 17 and 32 simultaneous typing users, the latency caused by the mixer will be around 300 ms and then passes both regulatory and human requirements.

Now, even if it passes the requirements, 32 is a very unrealistic number of simultaneous typing users. In audio conferences it is only possible to perceive one source at a time well. The benefit of enabling more is just for noticing that someone else want to say something. Requirements for this work are collected in draft-hellstrom-avtcore-multi-party-rtt-solutions-00, and there the performance requirements are set to be valid for up to 5 simultaneously transmitting users and the delay caused by the mixer to be less than 500 ms. I think we should design for these figures.

  *   And you mentioned these characteristics provide for smooth flow of text with acceptable latency from at least 32 sources simultaneously. Since the new packet format can support up to 16 sources per packet, the text from 32 sources will have be transmitted in turn. If my calculation is correct, with 300ms transmission interval and redundancy level 2, it will take 900ms (one primary + 2 redundant) for mixer to switch from first 16 sources to next 16 sources, so the delay is about 900ms. Is this the acceptable latency in your mind?
See discussions above. Maybe regulators need to say how many simultaneous users the requirements are for. I think 5 is a high and good figure even if the discussion above indicates 32 to be possible.
[YX] Yes I agree, at least for emergency type of service, I don't see a multi-party use case that requires more than 5 parties

Yes, I hope for a short period of discussions like yours that I appreciate to sort out the main principles, and then a period of the different levels of last calls and refinement editing. Even if one would hope for a more rapid progress, a realistic milestone for sending it to IESG has been set for February 2021. It would be good if various organisations who would reference it in their specifications (as 3GPP, ATIS, NENA, ETSI etc.) would take a look already now and assess if it is agreeable and on the right way.

[YX] Thanks for making effort on this

Thanks

Gunnar

  *

Regards,
Yong

From: Gunnar Hellström <[email-masked]><mailto:[email-masked]>
Sent: Thursday, May 21, 2020 12:39 AM
To: Yong Xin <[email-masked]><mailto:[email-masked]>; [email-masked]<mailto:[email-masked]>
Subject: Re: Question on multi-party RTT handling (draft-hellstrom-avtcore-multi-party-rtt-source-03)

Dear  Yong,

   sessions.

       0                   1                   2                   3

Thanks,

Gunnar

Thanks,
Yong

--

Gunnar Hellström

GHAccess

--

Gunnar Hellström

GHAccess

---

From: Gunnar Hellström <[email-masked]>
Date: Sat, 23 May 2020 15:24:27 +0200

Hi Yong, please see inline,

Den 2020-05-23 kl. 01:31, skrev Yong Xin:
>         redundancy level when transmitting texts from multiple
>         sources. If the different party in the same conference has
>         negotiated a different redundancy level, the mixer has to pick
>         the lowest level to use, right?
>
> users do not want to see their own transmitted text being received 
> from the mixer. The own text is displayed locally by the endpoints. If 
> the recipient does not support "text/rex", the mixer also need to do 
> the mixing for multi-party unaware endpoints using the "text/red" 
> format described in section 13.2.
>
> */[YX] Understood, the text mixer is providing N-1 mixing, similar to 
> audio mixing, so the user never receive their own transmitted text 
> from the mixer./*
>
/*[GH] */Yes. I see I have that specified for the multi-party unaware 
mixing in 13.2.2 by this sentence: "Text received from a participant 
SHOULD NOT be included intransmission to that participant." I suggest I 
include a similar sentence in 13.1 for the multi-party aware case.
> *//*
> */[YX] I agree what you said here. I think I’m little confused when 
> reading the following paragraph in section 3 of the spec. Let me put 
> an example, there’s a 3-party conference and all participants (A, B, 
> C) are conference-aware RTT terminals and support text/rex packet 
> format. User A, B, C negotiates different redundancy level 2, 3, 1 
> respectively. When mixer transmitting text (source from B & C) to user 
> A, what is the number of redundant generations should be used by the 
> mixer in the transmitted packet? Is it 2 or 1?
> /*
>
[GH] It is 1. I see my wording is confusing. I suggest to improve the 
yellow sentence fron the paragraph below to read: "/It//SHOULD be set to 
the minimum of the number declared by the two parties in the SDP 
exchange//."/

It can be discussed if the minimum of what the parties declared is the 
best choice. It is quite common that SDP parameters declare what the 
party wants to receive. In this case the number can be decided by the 
party knowing the network conditions in the network, or it can know it 
has decoding limitations and does not want to receive more than a 
specific number of generations in the packets. It may also have coding 
limitations, so that it cannot create more generations than itself can 
receive. That made me think that the resulting number of generations 
sent should be the minimum of what the two parties in each connection 
declared. I can change that to say that "It SHOULD be set to what the 
other party declared". What is actually used in the stream is found by 
dividing the number of data header entries with the number of members in 
the CSRC list.

This discussion is a bit theoretical, because the default is one primary 
and two redundant generations, and there are rarely any reason to 
deviate from that.

> *//*
>
> /   The number of redundant generations of T140blocks to include in/
>
> /   transmitted packets SHALL be deducted from the SDP negotiation. It/
>
> /SHOULD be set to the minimum of the number declared by the receiver/
>
> /and the transmitter//.  The same number of redundant generations MUST/
>
> /   be used for all sources in the transmissions.  The number of/
>
> /   generations sent to a receiver SHALL be the same during the whole/
>
> /   session unless it is modified by session renegotiation./
>
>       * But in case there’s one party has negotiated “text/rex”
>         without any redundancy level, does that mean mixer has to turn
>         of the redundancy for this conference? Does mixer need to
>         change the redundancy level up and down dynamically as user
>         joins or leaves the conference? Does mixer need to send
>         re-INVITE to re-negotiate the redundancy level with other
>         party when such change happens?
>
> From the logic above, the answers on these questions are: no. I 
> realize that an explanation should be inserted in the beginning of 
> section 3. "Actions at transmission by the mixer" to clarify that the 
> source for transmission from the mixer is clean text in separate 
> queues regardless of which format or protocol they used in the 
> individual receptions.
>
> */[YX] This is related to the above question. Some clarification in 
> the spec would be helpful./*
>
>       * In section 12, I noticed the 150cps recommendation is still
>         there and has been made as default value for the new packet
>         format, but the transmission interval is back to 300ms (the
>         recommended interval was 100ms in the old spec). I guess with
>         the new packet format, it is not required to use the shorter
>         transmission interval any more.
>
> packet to each receiver as soon as text has been received from a 
> source as long as the maximum number of characters per second 
> indicated by the recipient is not exceeded, and also the number of 
> packets sent per second to a recipient is kept under a specified 
> number.  This number SHALL be 10 if no other limit is applied for the 
> application.  The intention is to keep the latency introduced by the 
> mixer low."
>
> This is intended to create a balance between low latency and 
> protection against bursty packet loss. Even if the latency 
> requirements from real-time text users are much lower than from audio 
> and video users, a low latency is appreciated, and latency of over 2 
> seconds end-to-end creates conversation problems.  Therefore, this 
> paragraph about when to transmit will self-regulate to about 100 ms 
> packet interval from about 3 simultaneous typing sources.
>
> */[YX] So are you saying the recommended transmission interval in the 
> new spec is still 100ms, and the 300ms is actually the time that 
> covers 3 transmissions (one primary transmission plus 2 redundancy 
> transmissions, assuming redundancy level 2), is my understanding 
> correct? I guess this is better clarified in the new spec. The old 
> spec is much clear in this definition. /*
>
/*[GH]*/ The intention now is to not have a fixed interval, but have it 
variable between 100 and 300 ms. This is in order to not add delay by 
the mixer when it is possible to avoid it.

So, when the mixer receives a packet, it checks for each party it is 
sending to, if it already has sent 10 packets to that party during the 
last second. If not, it sends the just received text immediately on to 
that party together with whatever redundant text there was to be sent. 
If it already had sent 10 packets, then this text needs to wait in queue 
until 300 ms has passed from the latest transmission and then sent 
together with any other new text and the redundancy. Whoops, there was a 
logic error in that algorithm. It can allow a burst of 10 packets sent 
to one party with very short intervals, and then three more packets 
during the same second with 300 ms interval. That causes a risk for loss 
of text if a short burst loss happens when the 10 packets are sent. And 
it results in 13 packets sent when the limit was intended to be 10.

So, it is likely better to say in chapter 3:

" A "text/rex" transmitter SHOULD send packets distributed in time as 
long as there is something (new orredundant T140blocks) to transmit.The 
maximum transmission interval SHOULD then be 300 ms.It is RECOMMENDED to 
send a packet to a receiver as soon as new text to that receiver is 
available, as long as the time after the latest sent packet to the same 
receiver is more than 100 ms, and also the maximum character rate to the 
receiver is not exceeded. The intention is to keep the latency low while 
keeping a good protection against text loss in bursty packet loss 
conditions."

The introduced delay for up to 16 simultaneously sending users will be 
between 0 and 100 ms. I think it is good that a middlebox like this does 
not introduce more latency. I hope we can have a discussion if we 
instead get too high risk for text loss in case of bursty packet loss.  
With intensive participants the coverage for such loss is only 200-300 
ms, while at low traffic it is 600-900 ms.

> *//*
> Now, even if it passes the requirements, 32 is a very unrealistic 
> number of simultaneous typing users. In audio conferences it is only 
> possible to perceive one source at a time well. The benefit of 
> enabling more is just for noticing that someone else want to say 
> something. Requirements for this work are collected in 
> performance requirements are set to be valid for up to 5 
> simultaneously transmitting users and the delay caused by the mixer to 
> be less than 500 ms. I think we should design for these figures.
>
>       * And you mentioned these characteristics provide for smooth
>         flow of text with acceptable latency from at least 32 sources
>         simultaneously. Since the new packet format can support up to
>         16 sources per packet, the text from 32 sources will have be
>         transmission interval and redundancy level 2, it will take
>         900ms (one primary + 2 redundant) for mixer to switch from
>         first 16 sources to next 16 sources, so the delay is about
>         900ms. Is this the acceptable latency in your mind?
>
> */[YX] Yes I agree, at least for emergency type of service, I don’t 
> see a multi-party use case that requires more than 5 parties /*
>
[GH] I want to create a specification that is regarded sufficient for 
all realistic use cases. The most commonly foreseen emergency service 
use case has only two occsionally simultaneously transmitting users.

It is hard to imagine a use case with 5 simultaneously sending 
participants, where it would be important for a user to be able to read 
text from one source with less than one second delay.

It does not happen for audio. As soon as two talks, the result is 
usually not perceivable.(Except in the efforts you experience now in 
Corona-times, with a choir of individual conference participants trying 
to sing together.)

With video, you cannot concentrate on what more than one does. But the 
mixer can usually present up to 9 or 16 or so with maintained temporal 
resolution but less spatial resolution. The receiver selects one to 
perceive.

One application that could cause many simultaneous sources of text would 
be a conference with voice to text translations in many languages for a 
large audience. That will require a user interface at the receiving end 
that shows one selected text stream and hides everything else. I imagine 
that there are other mechanisms for that already. It cannot be the 
application we design for, but I dont mind if it will be possible.

This reasoning tells again that a goal of 5 simultaneous text sources 
with less than 500 ms delay is high but maybe realistic in some 
situation, and our solution that provides for 16 simultaneous 
transmitting users and 0-100 ms delay meets the requirement well. I do 
not think we will need to mention the 32 simultaneously transmitting 
users still meeting the requirements.

I will adjust the performance chapter and the introduction for this.

> [GH] Thanks.

---

From: Yong Xin <[email-masked]>
Date: Tue, 26 May 2020 17:12:53 +0000

Thanks Gunnar for the clarification and suggested change in next revision - they all look good to me

Regards,
Yong

From: Gunnar Hellström <[email-masked]>
Sent: Saturday, May 23, 2020 6:24 AM
To: Yong Xin <[email-masked]>
Cc: [email-masked]
Subject: Re: Question on multi-party RTT handling (draft-ietf-avtcore-multi-party-rtt-mix-02)

[YX] Understood, the text mixer is providing N-1 mixing, similar to audio mixing, so the user never receive their own transmitted text from the mixer.
[GH] Yes. I see I have that specified for the multi-party unaware mixing in 13.2.2 by this sentence: "Text received from a participant SHOULD NOT be included in transmission to that participant." I suggest I include a similar sentence in 13.1 for the multi-party aware case.

[GH] It is 1. I see my wording is confusing. I suggest to improve the yellow sentence fron the paragraph below to read: "It SHOULD be set to the minimum of the number declared by the two parties in the SDP exchange."

It can be discussed if the minimum of what the parties declared is the best choice. It is quite common that SDP parameters declare what the party wants to receive. In this case the number can be decided by the party knowing the network conditions in the network, or it can know it has decoding limitations and does not want to receive more than a specific number of generations in the packets. It may also have coding limitations, so that it cannot create more generations than itself can receive. That made me think that the resulting number of generations sent should be the minimum of what the two parties in each connection declared. I can change that to say that "It SHOULD be set to what the other party declared". What is actually used in the stream is found by dividing the number of data header entries with the number of members in the CSRC list.

This discussion is a bit theoretical, because the default is one primary and two redundant generations, and there are rarely any reason to deviate from that.

[GH] The intention now is to not have a fixed interval, but have it variable between 100 and 300 ms. This is in order to not add delay by the mixer when it is possible to avoid it.

So, when the mixer receives a packet, it checks for each party it is sending to, if it already has sent 10 packets to that party during the last second. If not, it sends the just received text immediately on to that party together with whatever redundant text there was to be sent. If it already had sent 10 packets, then this text needs to wait in queue until 300 ms has passed from the latest transmission and then sent together with any other new text and the redundancy. Whoops, there was a logic error in that algorithm. It can allow a burst of 10 packets sent to one party with very short intervals, and then three more packets during the same second with 300 ms interval. That causes a risk for loss of text if a short burst loss happens when the 10 packets are sent. And it results in 13 packets sent when the limit was intended to be 10.

" A "text/rex" transmitter SHOULD send packets distributed in time as long as there is something (new or  redundant T140blocks) to transmit.  The maximum transmission interval SHOULD then be 300 ms. It is RECOMMENDED to send a packet to a receiver as soon as new text  to that receiver is available, as long as the time after the latest sent packet to the same receiver is more than 100 ms, and also the maximum character rate to the receiver is not exceeded. The intention is to keep the latency low while keeping a good protection against text loss in bursty packet loss conditions."

The introduced delay for up to 16 simultaneously sending users will be between 0 and 100 ms. I think it is good that a middlebox like this does not introduce more latency. I hope we can have a discussion if we instead get too high risk for text loss in case of bursty packet loss.  With intensive participants the coverage for such loss is only 200-300 ms, while at low traffic it is 600-900 ms.

[GH] I want to create a specification that is regarded sufficient for all realistic use cases. The most commonly foreseen emergency service use case has only two occsionally simultaneously transmitting users.

It is hard to imagine a use case with 5 simultaneously sending participants, where it would be important for a user to be able to read text from one source with less than one second delay.

It does not happen for audio. As soon as two talks, the result is usually not perceivable.(Except in the efforts you experience now in Corona-times, with a choir of individual conference participants trying to sing together.)

With video, you cannot concentrate on what more than one does. But the mixer can usually present up to 9 or 16 or so with maintained temporal resolution but less spatial resolution. The receiver selects one to perceive.

One application that could cause many simultaneous sources of text would be a conference with voice to text translations in many languages for a large audience. That will require a user interface at the receiving end that shows one selected text stream and hides everything else. I imagine that there are other mechanisms for that already. It cannot be the application we design for, but I dont mind if it will be possible.

This reasoning tells again that a goal of 5 simultaneous text sources with less than 500 ms delay is high but maybe realistic in some situation, and our solution that provides for 16 simultaneous transmitting users and 0-100 ms delay meets the requirement well. I do not think we will need to mention the 32 simultaneously transmitting users still meeting the requirements.

--

Gunnar Hellström

GHAccess

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 27 May 2020 08:00:58 +0200

Thanks Yong for feedback,

We discussed performance requirements initially. I have checked the 
requirements from regulation outside of the emergency services. There, 
there is only a requirement to identify the source and be able to switch 
source smoothly in an ordely way between users taking turns properly.

In practice I know that there will be some simultaneous typing, at least 
for very brief indications, like "I want to say something" or "that 
suits me". So, good switching performance without much extra delay 
between 5 simultaneous typing users as the documented goal in 
draft-hellstrom-avtcore-multi-party-rtt-solutions-00 seems to be a safe 
and maybe a bit high requirement.

With the intended use for emergency services, the situation is similar 
there. I expect eager users in emergency to sometimes type longer 
explanations while they get instructions. But instead fewer extra 
persons jumping in with comments. So also there, switching without too 
much extra delay between a maximum of 5 simultaneous senders seem to be 
a safe and a bit high requirement.

Regards

Gunnar

Den 2020-05-26 kl. 19:12, skrev Yong Xin:
>
> Thanks Gunnar for the clarification and suggested change in next 
> revision – they all look good to me
>
> Regards,
>
> Yong
>
> *Sent:* Saturday, May 23, 2020 6:24 AM
> *To:* Yong Xin <[email-masked]>
> *Cc:* [email-masked]
> (draft-ietf-avtcore-multi-party-rtt-mix-02)
>
>
> 	
>
>           * The new payload format “text/rex” can be used with or
>             without redundancy. When redundancy is used, mixer has to
>             use the same redundancy level when transmitting texts from
>             multiple sources. If the different party in the same
>             conference has negotiated a different redundancy level,
>             the mixer has to pick the lowest level to use, right?
>
>     redundancy level agreed with each recipient. This is also because
>     the users do not want to see their own transmitted text being
>     received from the mixer. The own text is displayed locally by the
>     endpoints. If the recipient does not support "text/rex", the mixer
>     also need to do the mixing for multi-party unaware endpoints using
>     the "text/red" format  described in section 13.2.
>
>     */[YX] Understood, the text mixer is providing N-1 mixing, similar
>     to audio mixing, so the user never receive their own transmitted
>     text from the mixer./*
>
> */[GH] /*Yes. I see I have that specified for the multi-party unaware 
> SHOULD NOT be included in transmission to that participant." I suggest 
> I include a similar sentence in 13.1 for the multi-party aware case.
>
>     And the mixer must recover from loss in reception from each source
>     and create a queue of clean text from each source before composing
>     the packets for transmission. The mixer cannot just resend
>     received packet contents with redundancy, because the recovery
>     mechanism requires the sequence number gaps for loss detection,
>     and the mixer must create its own sequence number series in the
>     transmission.
>
>     */[YX] I agree what you said here. I think I’m little confused
>     when reading the following paragraph in section 3 of the spec. Let
>     me put an example, there’s a 3-party conference and all
>     participants (A, B, C) are conference-aware RTT terminals and
>     support text/rex packet format. User A, B, C negotiates different
>     redundancy level 2, 3, 1 respectively. When mixer transmitting
>     text (source from B & C) to user A, what is the number of
>     redundant generations should be used by the mixer in the
>     transmitted packet? Is it 2 or 1? /*
>
> yellow sentence fron the paragraph below to read: "/It//SHOULD be set 
> to the minimum of the number declared by the two parties in the SDP 
> exchange//."/
>
> dividing the number of data header entries with the number of members 
> in the CSRC list.
>
> This discussion is a bit theoretical, because the default is one 
> primary and two redundant generations, and there are rarely any reason 
> to deviate from that.
>
>     *//*
>
>             without any redundancy level, does that mean mixer has to
>             turn of the redundancy for this conference? Does mixer
>             need to change the redundancy level up and down
>             dynamically as user joins or leaves the conference? Does
>             mixer need to send re-INVITE to re-negotiate the
>             redundancy level with other party when such change happens?
>
>     section 3. "Actions at transmission by the mixer" to clarify that
>     the source for transmission from the mixer is clean text in
>     separate queues regardless of which format or protocol they used
>     in the individual receptions.
>
>
>     */[YX] This is related to the above question. Some clarification
>     in the spec would be helpful./*
>
>           * In section 12, I noticed the 150cps recommendation is
>             still there and has been made as default value for the new
>             packet format, but the transmission interval is back to
>             300ms (the recommended interval was 100ms in the old
>             spec). I guess with the new packet format, it is not
>             required to use the shorter transmission interval any more.
>
>     The transmission interval is mentioned in two paragraphs in
>     section 3. One saying that the default is 300 ms, the other saying:
>
>     "For multi-party operation, it is RECOMMENDED that the mixer sends
>     a packet to each receiver as soon as text has been received from a
>     number.  This number SHALL be 10 if no other limit is applied for
>     the application.  The intention is to keep the latency introduced
>     by the mixer low."
>
>     requirements from real-time text users are much lower than from
>     audio and video users, a low latency is appreciated, and latency
>     of over 2 seconds end-to-end creates conversation problems. 
>     Therefore, this paragraph about when to transmit will
>     self-regulate to about 100 ms packet interval from about 3
>     The 300 ms default assures that the remaining redundancy
>     transmissions will be sent even shortly after all sources have
>     stopped typing.
>
>     */[YX] So are you saying the recommended transmission interval in
>     the new spec is still 100ms, and the 300ms is actually the time
>     that covers 3 transmissions (one primary transmission plus 2
>     redundancy transmissions, assuming redundancy level 2), is my
>     understanding correct? I guess this is better clarified in the new
>     spec. The old spec is much clear in this definition. /*
>
> */[GH]/*The intention now is to not have a fixed interval, but have it 
> If it already had sent 10 packets, then this text needs to wait in 
> queue until 300 ms has passed from the latest transmission and then 
> sent together with any other new text and the redundancy. Whoops, 
> there was a logic error in that algorithm. It can allow a burst of 10 
> packets sent to one party with very short intervals, and then three 
> more packets during the same second with 300 ms interval. That causes 
> a risk for loss of text if a short burst loss happens when the 10 
> packets are sent. And it results in 13 packets sent when the limit was 
> intended to be 10.
>
> long as there is something (new or  redundant T140blocks) to transmit. 
> The maximum transmission interval SHOULD then be 300 ms. It is 
> RECOMMENDED to send a packet to a receiver as soon as new text  to 
> that receiver is available, as long as the time after the latest sent 
> packet to the same receiver is more than 100 ms, and also the maximum 
> character rate to the receiver is not exceeded. The intention is to 
> keep the latency low while keeping a good protection against text loss 
> in bursty packet loss conditions."
>
> between 0 and 100 ms. I think it is good that a middlebox like this 
> does not introduce more latency. I hope we can have a discussion if we 
> instead get too high risk for text loss in case of bursty packet 
> loss.  With intensive participants the coverage for such loss is only 
> 200-300 ms, while at low traffic it is 600-900 ms.
>
>     *//*
>
>     However, this algorithm may make the protection against bursty
>     loss weaker than with a steady 300 ms interval. With between 17
>     and 32 simultaneous typing users, the latency caused by the mixer
>     will be around 300 ms and then passes both regulatory and human
>     requirements.
>
>     number of simultaneous typing users. In audio conferences it is
>     only possible to perceive one source at a time well. The benefit
>     of enabling more is just for noticing that someone else want to
>     say something. Requirements for this work are collected in
>     draft-hellstrom-avtcore-multi-party-rtt-solutions-00, and there
>     the performance requirements are set to be valid for up to 5
>     simultaneously transmitting users and the delay caused by the
>     mixer to be less than 500 ms. I think we should design for these
>     figures.
>
>             flow of text with acceptable latency from at least 32
>             sources simultaneously. Since the new packet format can
>             support up to 16 sources per packet, the text from 32
>             sources will have be transmitted in turn. If my
>             calculation is correct, with 300ms transmission interval
>             and redundancy level 2, it will take 900ms (one primary +
>             2 redundant) for mixer to switch from first 16 sources to
>             next 16 sources, so the delay is about 900ms. Is this the
>             acceptable latency in your mind?
>
>     simultaneous users the requirements are for. I think 5 is a high
>     and good figure even if the discussion above indicates 32 to be
>     possible.
>     */[YX] Yes I agree, at least for emergency type of service, I
>     don’t see a multi-party use case that requires more than 5 parties /*
>
> participants, where it would be important for a user to be able to 
> read text from one source with less than one second delay.
>
> Corona-times, with a choir of individual conference participants 
> trying to sing together.)
>
> One application that could cause many simultaneous sources of text 
> would be a conference with voice to text translations in many 
> languages for a large audience. That will require a user interface at 
> the receiving end that shows one selected text stream and hides 
> everything else. I imagine that there are other mechanisms for that 
> already. It cannot be the application we design for, but I dont mind 
> if it will be possible.
>

---

From: Yong Xin <[email-masked]>
Date: Wed, 27 May 2020 15:32:52 +0000

Hi Gunnar,

Agreed, the mixer performance requirement of 5 simultaneous text sources with less than 500ms delay should be sufficient. BTW, do you expect to send the BCP document https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00 to IESG for approval around the same time (Feb/2021) as the standard track document https://tools.ietf.org/html/draft-ietf-avtcore-multi-party-rtt-mix-02?

Thanks,
Yong

From: Gunnar Hellström <[email-masked]>
Sent: Tuesday, May 26, 2020 11:01 PM
To: Yong Xin <[email-masked]>
Cc: [email-masked]
Subject: Re: Question on multi-party RTT handling (draft-ietf-avtcore-multi-party-rtt-mix-02)

We discussed performance requirements initially. I have checked the requirements from regulation outside of the emergency services. There, there is only a requirement to identify the source and be able to switch source smoothly in an ordely way between users taking turns properly.

In practice I know that there will be some simultaneous typing, at least for very brief indications, like "I want to say something" or "that suits me". So, good switching performance without much extra delay between 5 simultaneous typing users as the documented goal in  draft-hellstrom-avtcore-multi-party-rtt-solutions-00 seems to be a safe and maybe a bit high requirement.

With the intended use for emergency services, the situation is similar there. I expect eager users in emergency to sometimes type longer explanations while they get instructions. But instead fewer extra persons jumping in with comments. So also there, switching without too much extra delay between a maximum of 5 simultaneous senders seem to be a safe and a bit high requirement.

Regards

Gunnar

Regards,
Yong

From: Gunnar Hellström <[email-masked]><mailto:[email-masked]>
Sent: Saturday, May 23, 2020 6:24 AM
To: Yong Xin <[email-masked]><mailto:[email-masked]>
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: Question on multi-party RTT handling (draft-ietf-avtcore-multi-party-rtt-mix-02)

--

Gunnar Hellström

GHAccess

--

Gunnar Hellström

GHAccess

---

From: Gunnar Hellström <[email-masked]>
Date: Wed, 27 May 2020 19:39:31 +0200

Hi Yong,

Den 2020-05-27 kl. 17:32, skrev Yong Xin:
>
> Hi Gunnar,
>
> Agreed, the mixer performance requirement of 5 simultaneous text 
> sources with less than 500ms delay should be sufficient. BTW, do you 
> expect to send the BCP document 
> https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00 
> to IESG for approval around the same time (Feb/2021) as the standard 
> track document 
> https://tools.ietf.org/html/draft-ietf-avtcore-multi-party-rtt-mix-02?
>
The .....-solutions document has not yet gone through the procedure to 
be adopted as a working group draft. So, there is no current plan for 
its completion date.

I have included a few words about the performance requirements in next 
version of draft-ietf-avtcore-multi-party-rtt-mix 
<https://tools.ietf.org/html/draft-ietf-avtcore-multi-party-rtt-mix-02> 
now.

Are there any specific part of 
draft-hellstrom-avtcore-multi-party-rtt-solutions 
<https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00> 
that you see value of to see published?

Regards

Gunnar

> *Sent:* Tuesday, May 26, 2020 11:01 PM
> there is only a requirement to identify the source and be able to 
> switch source smoothly in an ordely way between users taking turns 
> properly.
>
> In practice I know that there will be some simultaneous typing, at 
> least for very brief indications, like "I want to say something" or 
> "that suits me". So, good switching performance without much extra 
> delay between 5 simultaneous typing users as the documented goal in 
> draft-hellstrom-avtcore-multi-party-rtt-solutions-00 seems to be a 
> safe and maybe a bit high requirement.
>
> much extra delay between a maximum of 5 simultaneous senders seem to 
> be a safe and a bit high requirement.
>
> Regards
>
> Gunnar
>
>     <mailto:[email-masked]>
>     *To:* Yong Xin <[email-masked]> <mailto:[email-masked]>
>     *Cc:* [email-masked] <mailto:[email-masked]>
>             Thanks for the quick response. The latest spec does
>             address my concern. I have some follow-up questions:
>
>                 without redundancy. When redundancy is used, mixer has
>                 to use the same redundancy level when transmitting
>                 texts from multiple sources. If the different party in
>                 the same conference has negotiated a different
>                 redundancy level, the mixer has to pick the lowest
>                 level to use, right?
>
>         The mixer should do separate mixing for each recipient, using
>         the redundancy level agreed with each recipient. This is also
>         because the users do not want to see their own transmitted
>         text being received from the mixer. The own text is displayed
>         locally by the endpoints. If the recipient does not support
>         "text/rex", the mixer also need to do the mixing for
>         multi-party unaware endpoints using the "text/red" format 
>         */[YX] Understood, the text mixer is providing N-1 mixing,
>         similar to audio mixing, so the user never receive their own
>         transmitted text from the mixer./*
>
>     */[GH] /*Yes. I see I have that specified for the multi-party
>     unaware mixing in 13.2.2 by this sentence: "Text received from a
>     participant SHOULD NOT be included in transmission to that
>     participant." I suggest I include a similar sentence in 13.1 for
>     the multi-party aware case.
>
>         And the mixer must recover from loss in reception from each
>         source and create a queue of clean text from each source
>         before composing the packets for transmission. The mixer
>         cannot just resend received packet contents with redundancy,
>         because the recovery mechanism requires the sequence number
>         gaps for loss detection, and the mixer must create its own
>         sequence number series in the transmission.
>
>         when reading the following paragraph in section 3 of the spec.
>         Let me put an example, there’s a 3-party conference and all
>         support text/rex packet format. User A, B, C negotiates
>         different redundancy level 2, 3, 1 respectively. When mixer
>         transmitting text (source from B & C) to user A, what is the
>         number of redundant generations should be used by the mixer in
>         the transmitted packet? Is it 2 or 1? /*
>
>     [GH] It is 1. I see my wording is confusing. I suggest to improve
>     the yellow sentence fron the paragraph below to read: "/ItSHOULD
>     be set to the minimum of the number declared by the two parties in
>     the SDP exchange."/
>
>     It can be discussed if the minimum of what the parties declared is
>     the best choice. It is quite common that SDP parameters declare
>     what the party wants to receive. In this case the number can be
>     decided by the party knowing the network conditions in the
>     network, or it can know it has decoding limitations and does not
>     want to receive more than a specific number of generations in the
>     packets. It may also have coding limitations, so that it cannot
>     create more generations than itself can receive. That made me
>     think that the resulting number of generations sent should be the
>     minimum of what the two parties in each connection declared. I can
>     change that to say that "It SHOULD be set to what the other party
>     declared". What is actually used in the stream is found by
>     dividing the number of data header entries with the number of
>     members in the CSRC list.
>
>     primary and two redundant generations, and there are rarely any
>     reason to deviate from that.
>
>         *//*
>
>               * But in case there’s one party has negotiated
>                 “text/rex” without any redundancy level, does that
>                 mean mixer has to turn of the redundancy for this
>                 conference? Does mixer need to change the redundancy
>                 level up and down dynamically as user joins or leaves
>                 the conference? Does mixer need to send re-INVITE to
>                 re-negotiate the redundancy level with other party
>                 when such change happens?
>
>         From the logic above, the answers on these questions are: no.
>         I realize that an explanation should be inserted in the
>         beginning of section 3. "Actions at transmission by the mixer"
>         to clarify that the source for transmission from the mixer is
>         clean text in separate queues regardless of which format or
>         protocol they used in the individual receptions.
>
>
>
>         */[YX] This is related to the above question. Some
>         clarification in the spec would be helpful../*
>
>                 still there and has been made as default value for the
>                 new packet format, but the transmission interval is
>                 back to 300ms (the recommended interval was 100ms in
>                 the old spec). I guess with the new packet format, it
>                 is not required to use the shorter transmission
>                 interval any more.
>
>         "For multi-party operation, it is RECOMMENDED that the mixer
>         sends a packet to each receiver as soon as text has been
>         received from a source as long as the maximum number of
>         characters per second indicated by the recipient is not
>         exceeded, and also the number of packets sent per second to a
>         recipient is kept under a specified number..  This number
>         SHALL be 10 if no other limit is applied for the application. 
>         The intention is to keep the latency introduced by the mixer low."
>
>         requirements from real-time text users are much lower than
>         from audio and video users, a low latency is appreciated, and
>         latency of over 2 seconds end-to-end creates conversation
>         problems.  Therefore, this paragraph about when to transmit
>         will self-regulate to about 100 ms packet interval from about
>         3 simultaneous typing sources.
>
>         */[YX] So are you saying the recommended transmission interval
>         in the new spec is still 100ms, and the 300ms is actually the
>         time that covers 3 transmissions (one primary transmission
>         plus 2 redundancy transmissions, assuming redundancy level 2),
>         is my understanding correct? I guess this is better clarified
>         in the new spec. The old spec is much clear in this definition. /*
>
>     */[GH]/*The intention now is to not have a fixed interval, but
>     have it variable between 100 and 300 ms. This is in order to not
>     add delay by the mixer when it is possible to avoid it.
>
>     So, when the mixer receives a packet, it checks for each party it
>     is sending to, if it already has sent 10 packets to that party
>     during the last second. If not, it sends the just received text
>     immediately on to that party together with whatever redundant text
>     there was to be sent. If it already had sent 10 packets, then this
>     text needs to wait in queue until 300 ms has passed from the
>     latest transmission and then sent together with any other new text
>     and the redundancy. Whoops, there was a logic error in that
>     algorithm. It can allow a burst of 10 packets sent to one party
>     with very short intervals, and then three more packets during the
>     same second with 300 ms interval. That causes a risk for loss of
>     text if a short burst loss happens when the 10 packets are sent.
>     And it results in 13 packets sent when the limit was intended to
>     be 10.
>
>     " A "text/rex" transmitter SHOULD send packets distributed in time
>     as long as there is something (new or  redundant T140blocks) to
>     transmit.  The maximum transmission interval SHOULD then be 300
>     ms. It is RECOMMENDED to send a packet to a receiver as soon as
>     new text  to that receiver is available, as long as the time after
>     the latest sent packet to the same receiver is more than 100 ms,
>     and also the maximum character rate to the receiver is not
>     exceeded. The intention is to keep the latency low while keeping a
>     good protection against text loss in bursty packet loss conditions."
>
>     The introduced delay for up to 16 simultaneously sending users
>     will be between 0 and 100 ms. I think it is good that a middlebox
>     like this does not introduce more latency. I hope we can have a
>     discussion if we instead get too high risk for text loss in case
>     of bursty packet loss.  With intensive participants the coverage
>     for such loss is only 200-300 ms, while at low traffic it is
>     600-900 ms.
>
>         *//*
>
>         loss weaker than with a steady 300 ms interval. With between
>         17 and 32 simultaneous typing users, the latency caused by the
>         mixer will be around 300 ms and then passes both regulatory
>         and human requirements.
>
>         Now, even if it passes the requirements, 32 is a very
>         unrealistic number of simultaneous typing users. In audio
>         conferences it is only possible to perceive one source at a
>         time well. The benefit of enabling more is just for noticing
>         that someone else want to say something. Requirements for this
>         work are collected in
>         draft-hellstrom-avtcore-multi-party-rtt-solutions-00, and
>         there the performance requirements are set to be valid for up
>         to 5 simultaneously transmitting users and the delay caused by
>         the mixer to be less than 500 ms. I think we should design for
>         these figures.
>
>               * And you mentioned these characteristics provide for
>                 smooth flow of text with acceptable latency from at
>                 least 32 sources simultaneously. Since the new packet
>                 format can support up to 16 sources per packet, the
>                 text from 32 sources will have be transmitted in turn.
>                 If my calculation is correct, with 300ms transmission
>                 interval and redundancy level 2, it will take 900ms
>                 (one primary + 2 redundant) for mixer to switch from
>                 first 16 sources to next 16 sources, so the delay is
>                 about 900ms. Is this the acceptable latency in your mind?
>
>         simultaneous users the requirements are for. I think 5 is a
>         high and good figure even if the discussion above indicates 32
>         to be possible.
>     [GH] I want to create a specification that is regarded sufficient
>     for all realistic use cases. The most commonly foreseen emergency
>     service use case has only two occsionally simultaneously
>     transmitting users.
>
>     usually not perceivable.(Except in the efforts you experience now
>     in Corona-times, with a choir of individual conference
>     participants trying to sing together.)
>
>     With video, you cannot concentrate on what more than one does. But
>     the mixer can usually present up to 9 or 16 or so with maintained
>     temporal resolution but less spatial resolution. The receiver
>     selects one to perceive.
>
>     languages for a large audience. That will require a user interface
>     at the receiving end that shows one selected text stream and hides
>     everything else. I imagine that there are other mechanisms for
>     that already. It cannot be the application we design for, but I
>     dont mind if it will be possible.
>
>     This reasoning tells again that a goal of 5 simultaneous text
>     sources with less than 500 ms delay is high but maybe realistic in
>     some situation, and our solution that provides for 16 simultaneous
>     transmitting users and 0-100 ms delay meets the requirement well.
>     I do not think we will need to mention the 32 simultaneously
>     transmitting users still meeting the requirements.
>
> _______________________________________________
> Audio/Video Transport Core Maintenance
> [email-masked]
> https://www.ietf.org/mailman/listinfo/avt

-- 

+ + + + + + + + + + + + + +

Gunnar Hellström
Omnitor
[email-masked]
+46 708 204 288

---

From: Yong Xin <[email-masked]>
Date: Wed, 27 May 2020 18:35:44 +0000

Hi Gunnar,

If you have clarified the mixer performance requirements in the next version of  draft-ietf-avtcore-multi-party-rtt-mix<https://tools.ietf.org/html/draft-ietf-avtcore-multi-party-rtt-mix-02>, then I'm fine.

Thanks,
Yong

From: avt <[email-masked]> On Behalf Of Gunnar Hellström
Sent: Wednesday, May 27, 2020 10:40 AM
To: [email-masked]
Subject: Re: [AVTCORE] Question on multi-party RTT handling (draft-ietf-avtcore-multi-party-rtt-mix-02)

Hi Yong,

The .....-solutions document has not yet gone through the procedure to be adopted as a working group draft. So, there is no current plan for its completion date.

I have included a few words about the performance requirements in next version of draft-ietf-avtcore-multi-party-rtt-mix<https://tools.ietf.org/html/draft-ietf-avtcore-multi-party-rtt-mix-02> now.

Are there any specific part of draft-hellstrom-avtcore-multi-party-rtt-solutions<https://tools.ietf.org/html/draft-hellstrom-avtcore-multi-party-rtt-solutions-00> that you see value of to see published?

Regards

Gunnar

Thanks,
Yong

Regards

Gunnar

Regards,
Yong

>From the logic above, the answers on these questions are: no. I realize that an explanation should be inserted in the beginning of section 3. "Actions at transmission by the mixer" to clarify that t…

[YX] This is related to the above question. Some clarification in the spec would be helpful..

"For multi-party operation, it is RECOMMENDED that the mixer sends a packet to each receiver as soon as text has been received from a source as long as the maximum number of characters per second indicated by the recipient is not exceeded, and also the number of packets sent per second to a recipient is kept under a specified number..  This number SHALL be 10 if no other limit is applied for the application.  The intention is to keep the latency introduced by the mixer low."

--

Gunnar Hellström

GHAccess

--

Gunnar Hellström

GHAccess

[email-masked]<mailto:[email-masked]>

--

Gunnar Hellström

Omnitor

[email-masked]<mailto:[email-masked]>

+46 708 204 288