Subject: [tsvwg] MP-DCCP for UDP multipath transport

From: <[email-masked]>
Date: Mon, 16 Nov 2020 12:52:04 +0000

Dear all,

as the TSVWG chairs suggested in the IETF 109 session, I continue the discussion on the mailinglist.

For all who were not able to attend, the presented slides are available under https://datatracker.ietf.org/meeting/109/materials/slides-109-tsvwg-sessa-61-dccp-extensions-for-multipath-operation-00, with a main focus on the prototype implementation. This prototype leverages MP-DCCP between a mobile handset and a public aggregation point, for giving multipath/multi-access benefit to at least UDP traffic - complementing MPTCP - when communicating with the Internet. Due to the MP-DCCP encapsulation framework, support is not restricted to UDP and it's even possible to provide the multipath service to IP or Ethernet traffic. Packet based scheduling, as well as flow based scheduling is in scope of the prototype/drafts and explained together with results as part of the above mentioned presentation.

Seamless handover for reliability and path/access aggregation for high speeds are supported by design, fitting for example excellently into the 3GPP ATSSS scope, leveraging:
https://tools.ietf.org/html/draft-amend-tsvwg-multipath-dccp-03
https://tools.ietf.org/html/draft-amend-tsvwg-multipath-framework-mpdccp-01
https://tools.ietf.org/html/draft-amend-tsvwg-dccp-udp-header-conversion-01

and general scheduling and re-ordering concepts from
https://tools.ietf.org/html/draft-amend-iccrg-multipath-reordering-01
https://tools.ietf.org/html/draft-bonaventure-iccrg-schedulers-01

Especially for 3GPP ATSSS purposes, this very dedicated and simple MP-DCCP framework provides an alternative over the currently discussed MP-QUIC (+DATAGRAM + MASQUE), which is from an IETF perspective much broader in scope and not dedicated to ATSSS purposes.

However MP-DCCP is not limited to ATSSS and can be applied end-to-end with or without encapsulation.

So I encourage you to read through the drafts, give feedback and discuss the potential roadmap towards WG adoption.

Br

Markus

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Nov 2020 15:59:07 +0100

> On 16. Nov 2020, at 13:52, [email-masked] wrote:
> 
> Dear all,
>  
Hi Markus,

I have two questions:

1. You stated that you are using CCID2. This means that you are adding to handling of UDP
  packets a congestion control. How does this interact with a congestion control provided by
  the application?

2. Considering that you are fine with a tunneling protocol that provides a congestion control,
  what are the benefits of using DCCP instead of using SCTP with UDP encapsulation? You can
  use PR-SCTP with the "don't retransmit" policy and could also let the reordering done by
  SCTP. Your failover scenario should be supported by all SCTP implementations. The
  load-sharing would require some standardisation effort.

Best regards
Michael
>  
> For all who were not able to attend, the presented slides are available underhttps://datatracker.ietf.org/meeting/109/materials/slides-109-tsvwg-sessa-61-dccp-extensions-for-multipath-operation-00, with a main focus on the prototype implementation. This prototype leverages MP-DCCP between a mobile handset and a public aggregation point, for giving multipath/multi-access benefit to at least UDP traffic - complementing MPTCP - when communicating with the Internet. Due to the MP-DCCP encapsulation framework, support is not restricted to UDP and it’s even possible to provide the multipath service to IP or Ethernet traffic. Packet based scheduling, as well as flow based scheduling is in scope of the prototype/drafts and explained together with results as part of the above mentioned presentation.
>

---

From: <[email-masked]>
Date: Mon, 16 Nov 2020 15:43:47 +0000

Hi Michael,

inline

> -----Original Message-----
> From: Michael Tuexen <[email-masked]>
> Sent: Montag, 16. November 2020 15:38
> To: Amend, Markus <[email-masked]>
> Subject: Re: [tsvwg] MP-DCCP for UDP multipath transport
> 

Yes, and moreover we also implemented BBRv1 into the DCCP CCID framework. The interaction between CCs is given and first results were published a while ago in https://arxiv.org/pdf/1907.04567.pdf based on NADA as application CC. More recent results are part of our presentation at ICCRG on Friday, discussing a broader mix of CCs. Probably there is a trend not to use packet loss based CC for the tunnel.

Two points here:
1. Nested CC is a broader topic beyond MP-DCCP and also valid e.g. for MASQUE. Maybe something which should be independently investigated in ICCRG?
2. Solving efficiently out-of-order aggregated multipath packet delivery is equally important and often overlooked in favor of CC in CC.

>    load-sharing would require some standardisation effort.

Excellent this question :-) I fully agree with you, that SCTP can provide similar mechanisms. Some years ago we developed, in parallel to our MP-DCCP prototype, a SCTP based prototype leveraging PR-, PF- and CMT-SCTP. The results looked very promising and we proposed this to 3GPP ATSSS (ftp://3gpp.org/Email_Discussions/SA2/Archive/2018-10/Draft%2023793-070_rm.doc, 6.1.7.4), but it was rejected.

After that we gave up the SCTP development in favor of DCCP for several reasons:
-  bad/no Linux support for CMT-SCTP at this time
-  No real unreliable delivery is given. Even PR-SCTP requires a reliable "FORWARD-TSN" to become unreliable.
-  UDP encapsulation overhead

> Best regards
> Michael
> 
> >
> > For all who were not able to attend, the presented slides are available
> u==[=[nderhttps://datatracker.ietf.org/meeting/109/materials/slides-109-tsvwg-
> sessa-61-dccp-extensions-for-multipath-operation-00, with a main focus on the
> prototype implementation. This prototype leverages MP-DCCP between a
> mobile handset and a public aggregation point, for giving multipath/multi-access
> benefit to at least UDP traffic - complementing MPTCP - when communicating
> with the Internet. Due to the MP-DCCP encapsulation framework, support is not
> restricted to UDP and it’s even possible to provide the multipath service to IP or
> Ethernet traffic. Packet based scheduling, as well as flow based scheduling is in
> scope of the prototype/drafts and explained together with results as part of the
> above mentioned presentation.
> >
> > Seamless handover for reliability and path/access aggregation for high speeds
> are supported by design, fitting for example excellently into the 3GPP ATSSS
> scope, leveraging:
> > Especially for 3GPP ATSSS purposes, this very dedicated and simple MP-DCCP
> framework provides an alternative over the currently discussed MP-QUIC
> (+DATAGRAM + MASQUE), which is from an IETF perspective much broader in
> scope and not dedicated to ATSSS purposes.
> >

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Nov 2020 17:05:30 +0100

> On 16. Nov 2020, at 16:43, [email-masked] wrote:
> 
> Hi Michael,
> 
> inline
> 
I agree that this is not specific to the scenario you describe, but a broader issue.
> -  bad/no Linux support for CMT-SCTP at this time
Compared to MP-DCCP support, I don't think there is much of a difference. I guess it would be implemented
in Linux, if the corresponding specification could be finished (Linux recently got support for UDP encapsulation.). However, that is blocked up to now in TSVWG.
Since I'm not following 3GPP:
Is availability on Linux a requirement for 3GPP?
> -  No real unreliable delivery is given. Even PR-SCTP requires a reliable "FORWARD-TSN" to become unreliable.
How does that impact the user experience? User messages can be delivered as soon as possible to the application.
> -  UDP encapsulation overhead
True. It is an 8 byte overhead.

Best regards
Michael

---

From: <[email-masked]>
Date: Mon, 16 Nov 2020 16:21:00 +0000

> Sent: Montag, 16. November 2020 17:06
> Cc: [email-masked]
> >>> as the TSVWG chairs suggested in the IETF 109 session, I continue the
> discussion
> >> on the mailinglist.
> >> Hi Markus,
> >>
> >> I have two questions:
> >>
> >>   packets a congestion control. How does this interact with a congestion
> control
> >> provided by
> >>   the application?
> >
> > Yes, and moreover we also implemented BBRv1 into the DCCP CCID framework.
> The interaction between CCs is given and first results were published a while ago
> in https://arxiv.org/pdf/1907.04567.pdf based on NADA as application CC. More
> recent results are part of our presentation at ICCRG on Friday, discussing a
> broader mix of CCs. Probably there is a trend not to use packet loss based CC for
> the tunnel.
> >
> > Two points here:
> > 1. Nested CC is a broader topic beyond MP-DCCP and also valid e.g. for
> MASQUE. Maybe something which should be independently investigated in
> ICCRG?
> > 2. Solving efficiently out-of-order aggregated multipath packet delivery is
> equally important and often overlooked in favor of CC in CC.
> >
> >>
> >>   use PR-SCTP with the "don't retransmit" policy and could also let the
> reordering
> >> done by
> >>   SCTP. Your failover scenario should be supported by all SCTP
> implementations.
> >> The
> > Excellent this question :-) I fully agree with you, that SCTP can provide similar
> mechanisms. Some years ago we developed, in parallel to our MP-DCCP
> prototype, a SCTP based prototype leveraging PR-, PF- and CMT-SCTP. The
> results looked very promising and we proposed this to 3GPP ATSSS
> (ftp://3gpp.org/Email_Discussions/SA2/Archive/2018-10/Draft%2023793-
> 070_rm.doc, 6.1.7.4), but it was rejected.
> >
> in Linux, if the corresponding specification could be finished (Linux recently got
> support for UDP encapsulation.). However, that is blocked up to now in TSVWG.

No, but it was a requirement for our DT internal prototype environment. With a Linux reference implementation of (MP-)DCCP it was much simpler for us to move forward and implement it for example in an Android device.
Please don't forget, that was a decision we took in 2017.

However, -just being proactively here - we currently should not end up with the question:  Should we move forward exclusively with MP-DCCP or a SCTP based approach. Both has from my view value and MP-DCCP is first of all only a multipath extension for DCCP. Using encapsulation on top makes it capable of transporting any traffic. It's the same as with SCTP, QUIC and TCP.

> soon as possible to the application.

Please correct me if I'm wrong, but as long as the FORWARD TSN has not be received, head of line blocking occurs, right?

> >>> https://tools.ietf.org/html/draft-amend-tsvwg-multipath-framework-
> mpdccp-
> >> 01
> >>> https://tools.ietf.org/html/draft-amend-tsvwg-dccp-udp-header-
> conversion-
> >> 01
> >>>
> >>>
> >>> However MP-DCCP is not limited to ATSSS and can be applied end-to-end
> with
> >> or without encapsulation.
> >>>

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Nov 2020 17:50:54 +0100

> On 16. Nov 2020, at 17:21, [email-masked] wrote:
> 
TCP might be a problem with strict ordering. But SCTP, QUIC, and DCCP could be used the same way.
The only difference is that SCTP does support multihoming, for QUIC and DCCP it is a yet to
be specified extension, QUIC needs unreliability, and all need a way of load sharing. But then
all of them could be used.
> Please correct me if I'm wrong, but as long as the FORWARD TSN has not be received, head of line blocking occurs, right?
I don't see why. The receiver can deliver user messages as soon as the ordering constraints can
be fulfilled. So if you send all user messages as unordered messages (like on UDP or DCCP),
no buffering for reordering is required. So no HOL blocking.

Best regards
Michael

---

From: <[email-masked]>
Date: Mon, 16 Nov 2020 17:46:01 +0000

> Sent: Montag, 16. November 2020 17:51
> > No, but it was a requirement for our DT internal prototype environment. With a
> Linux reference implementation of (MP-)DCCP it was much simpler for us to
> move forward and implement it for example in an Android device.
> > However, -just being proactively here - we currently should not end up with
> the question:  Should we move forward exclusively with MP-DCCP or a SCTP
> based approach. Both has from my view value and MP-DCCP is first of all only a
> multipath extension for DCCP. Using encapsulation on top makes it capable of
> transporting any traffic. It's the same as with SCTP, QUIC and TCP.
> TCP might be a problem with strict ordering.

Agree

> But SCTP, QUIC, and DCCP could be
> used the same way.
> >> How does that impact the user experience? User messages can be delivered
> as
> >> soon as possible to the application.
> >
> > Please correct me if I'm wrong, but as long as the FORWARD TSN has not be
> received, head of line blocking occurs, right?

Yep, I identified we argued from different perspectives. I agree, setting the U-Bit will bypass any re-ordering functionality from SCTP and move packets forward immediately to the application layer. That would give support for steering and switching (seamless handover), however, for CMT-SCTP the SCTP re-ordering function might be kept and then we step into the FORWARD-TSN trap with HoL. Unless a re-ordering function is specified outside of the SCTP.
In contrast to this, our MP-DCCP Linux reference implementation is shipped with a modular re-ordering scheme, which easily let define and use various re-ordering logics. I hope we can publish the Linux implementation soon.

> >>>> with the Internet. Due to the MP-DCCP encapsulation framework, support
> is
> >> not
> >>>> restricted to UDP and it’s even possible to provide the multipath service to
> IP
> >> or
> >>>> scope of the prototype/drafts and explained together with results as part
> of
> >> the
> >>>>> So I encourage you to read through the drafts, give feedback and discuss
> the
> >>>> potential roadmap towards WG adoption.
> >>>>>
> >>>>> Br
> >>>>>
> >>>>> Markus
> >>>
> >

---

From: Michael Tuexen <[email-masked]>
Date: Mon, 16 Nov 2020 19:39:44 +0100

> On 16. Nov 2020, at 18:46, [email-masked] wrote:
> 
As far as I understand, DCCP does not provide an ordering preserving service. Isn't it a UDP like service?
> In contrast to this, our MP-DCCP Linux reference implementation is shipped with a modular re-ordering scheme, which easily let define and use various re-ordering logics. I hope we can publish the L…
So you are adding something to DCCP? You could do that also as a shim on top of SCTP using the U-bit.

Best regards
Michael

---

From: <[email-masked]>
Date: Mon, 16 Nov 2020 19:04:30 +0000

> Sent: Montag, 16. November 2020 19:40
> >>>> The interaction between CCs is given and first results were published a
> while
> >> ago
> >>>> broader mix of CCs. Probably there is a trend not to use packet loss based
> CC
> >> for
> >>>> the tunnel.
> >>>>>
> >>>>> Two points here:
> >>>> Compared to MP-DCCP support, I don't think there is much of a difference.
> I
> >>>> guess it would be implemented
> >> The only difference is that SCTP does support multihoming, for QUIC and
> DCCP it
> >> is a yet to
> >> I don't see why. The receiver can deliver user messages as soon as the
> ordering
> >> constraints can
> >> be fulfilled. So if you send all user messages as unordered messages (like on
> UDP
> >> or DCCP),
> > Yep, I identified we argued from different perspectives. I agree, setting the U-
> Bit will bypass any re-ordering functionality from SCTP and move packets forward
> immediately to the application layer. That would give support for steering and
> switching (seamless handover), however, for CMT-SCTP the SCTP re-ordering
> function might be kept and then we step into the FORWARD-TSN trap with HoL.
> Unless a re-ordering function is specified outside of the SCTP.

That's true

> > In contrast to this, our MP-DCCP Linux reference implementation is shipped
> with a modular re-ordering scheme, which easily let define and use various re-
> ordering logics. I hope we can publish the Linux implementation soon.

Yes, that's what I wrote. However, with MP-DCCP we could think about defining a modular re-ordering scheme directly as part of the protocol, at least the signaling/negotiation for it. So no shim layer is required.

In the meantime I also remember another experienced drawback from our SCTP prototype. I think it was quit inflexible when new communication paths came up during the lifetime of a SCTP session, a scenario often happens in mobile use cases. Only with an INIT chunk, communication paths can be negotiated once during the handshake. Any new IP addresses/paths, e.g. while changing from one Wi-Fi AP to another, requires a re-establishment of the SCTP session.

I think for MP-DCCP we can resemble a lot of the well proved handshake procedures and address/path updates from MPTCP. 

> >>>>>> u==[=[nderhttps://datatracker.ietf.org/meeting/109/materials/slides-
> 109-
> >>>> tsvwg-
> >>>>>> sessa-61-dccp-extensions-for-multipath-operation-00, with a main focus
> on
> >>>> the
> >>>>>> mobile handset and a public aggregation point, for giving
> multipath/multi-
> >>>> access
> >>>>>> Ethernet traffic. Packet based scheduling, as well as flow based
> scheduling
> >> is in
> >>>>>> (+DATAGRAM + MASQUE), which is from an IETF perspective much
> broader
> >> in

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 16 Nov 2020 17:09:09 -0600

Just chiming in here ...

On Mon, Nov 16, 2020 at 10:21 AM <[email-masked]> wrote:

>
> However, -just being proactively here - we currently should not end up
> with the question:  Should we move forward exclusively with MP-DCCP or a
> SCTP based approach. Both has from my view value and MP-DCCP is first of
> all only a multipath extension for DCCP. Using encapsulation on top makes
> it capable of transporting any traffic. It's the same as with SCTP, QUIC
> and TCP.
>

Whether we end up with the MP-DCCP-or-multipath SCTP question or not, we
shouldn't start out with that question.

SCTP and DCCP really are different protocols. It's probably best if we
don't make them compete head to head.

Once we have a better understanding of the potential for multipath
operation of each protocol, we might have a conversation about whether we
need both. Or we might not need to have that conversation at all.

But let's continue the kind of discussion that's happening in this thread.
It's quite encouraging, at least to me.

And thanks, Markus and Michael, for having that discussion on-list.

Best,

Spencer

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 16 Nov 2020 17:13:20 -0600

Hi, Markus,

On Mon, Nov 16, 2020 at 9:44 AM <[email-masked]> wrote:

> >    packets a congestion control. How does this interact with a
> congestion control
> > provided by
> >    the application?
>
> while ago in https://arxiv.org/pdf/1907.04567.pdf based on NADA as
> application CC. More recent results are part of our presentation at ICCRG
> on Friday, discussing a broader mix of CCs. Probably there is a trend not
> to use packet loss based CC for the tunnel.
>

Just to make sure I understand - do you mean "not to use packet loss based
CC for the tunnel" to mean CC inside the tunnel, or CC outside the tunnel?

> MASQUE. Maybe something which should be independently investigated in ICCRG?
> 2. Solving efficiently out-of-order aggregated multipath packet delivery
> is equally important and often overlooked in favor of CC in CC.

I'd agree with both of these points.

Best,

Spencer

---

From: Martin Duke <[email-masked]>
Date: Mon, 16 Nov 2020 16:45:32 -0800

Hi Markus,

I should note that MASQUE is specifically chartered to provide
recommendations for cc-in-cc without designing a new protocol or starting
any research projects. The draft probably ought to say *something.*

Header conversion and multipath seem uncontroversial to me, but
encapsulation may raise some concerns. Can you tell me how separable the
drafts are? Would there be any point in adopting two of them, or is
encapsulation the use case that drives all the interest in these changes?

On Mon, Nov 16, 2020 at 11:04 AM <[email-masked]> wrote:

> > >>>>>> 1. You stated that you are using CCID2. This means that you are
> adding to
> > >>>>>> handling of UDP
> congestion
> > >>>> in https://arxiv.org/pdf/1907.04567.pdf based on NADA as
> application CC.
> > >> More
> > >>>> I agree that this is not specific to the scenario you describe, but
> a broader
> > >> issue.
> > >>>>>> 2. Considering that you are fine with a tunneling protocol that
> provides a
> > >>>>>> congestion control,
> > >>>>> Excellent this question :-) I fully agree with you, that SCTP can
> provide
> > similar
> > >>>> (
> ftp://3gpp.org/Email_Discussions/SA2/Archive/2018-10/Draft%2023793-
> > >>>>> After that we gave up the SCTP development in favor of DCCP for
> several
> > >>>> reasons:
> > >>>> in Linux, if the corresponding specification could be finished
> (Linux recently
> > >> got
> > >>>> support for UDP encapsulation.). However, that is blocked up to now
> in
> > >> TSVWG.
> > >>> No, but it was a requirement for our DT internal prototype
> environment.
> > With a
> > >> based approach. Both has from my view value and MP-DCCP is first of
> all only
> > a
> > >> be specified extension, QUIC needs unreliability, and all need a way
> of load
> > >> sharing. But then
> > >>>>> -  No real unreliable delivery is given. Even PR-SCTP requires a
> reliable
> > >>>> "FORWARD-TSN" to become unreliable.
> > As far as I understand, DCCP does not provide an ordering preserving
> service.
> > Isn't it a UDP like service?
>
> That's true
>
> > So you are adding something to DCCP? You could do that also as a shim on
> top of
> > SCTP using the U-bit.
> >
>
> Yes, that's what I wrote. However, with MP-DCCP we could think about
> defining a modular re-ordering scheme directly as part of the protocol, at
> least the signaling/negotiation for it. So no shim layer is required.
>
>
> In the meantime I also remember another experienced drawback from our SCTP
> prototype. I think it was quit inflexible when new communication paths came
> up during the lifetime of a SCTP session, a scenario often happens in
> mobile use cases. Only with an INIT chunk, communication paths can be
> negotiated once during the handshake. Any new IP addresses/paths, e.g.
> while changing from one Wi-Fi AP to another, requires a re-establishment of
> the SCTP session.
>
> I think for MP-DCCP we can resemble a lot of the well proved handshake
> procedures and address/path updates from MPTCP.
>
> > Best regards
> > Michael
> > >
> > >> Best regards
> > >> Michael
> > >>>
> > >>>>>> u==[=[nderhttps://
> datatracker.ietf.org/meeting/109/materials/slides-
> > 109-
> > >>>> tsvwg-
> > >>>>>> restricted to UDP and it’s even possible to provide the multipath
> service
> > to
> > >> IP
> > >>>> or
> > >>>>>> scope of the prototype/drafts and explained together with results
> as
> > part
> > >> of
> > >>>> the
> > >>>>>>> Seamless handover for reliability and path/access aggregation
> for high
> > >>>> speeds
> > >>>>>> are supported by design, fitting for example excellently into the
> 3GPP
> > >> ATSSS
> > >>>>>> scope, leveraging:
> > >>>>>>> Especially for 3GPP ATSSS purposes, this very dedicated and
> simple MP-
> > >> DCCP
> > >>>>>>> However MP-DCCP is not limited to ATSSS and can be applied
> end-to-
> > end
> > >>>> with

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 07:59:12 +0000

From: Spencer Dawkins at IETF <[email-masked]> 
Sent: Dienstag, 17. November 2020 00:13
To: Amend, Markus <[email-masked]>
Cc: Michael Tuexen <[email-masked]>; tsvwg <[email-masked]>
Subject: Re: [tsvwg] MP-DCCP for UDP multipath transport

> > On Mon, Nov 16, 2020 at 9:44 AM <mailto:[email-masked]> wrote:
> > Hi Michael,

> From: Michael Tuexen <mailto:[email-masked]>
> To: Amend, Markus <mailto:[email-masked]>
> > On 16. Nov 2020, at 13:52, mailto:[email-masked] wrote:
> >
> > Dear all,
> >

> Just to make sure I understand - do you mean "not to use packet loss based CC for the tunnel" to mean CC inside the tunnel, or CC outside the tunnel?

Not sure if we have identical understanding of inside/outside, that's why I explain it in different words. I referred to the CC which is used from the DCCP tunnel connection. I do not refer to the CC used by piggybacked payload.

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 08:07:48 +0000

Hi Martin,

Good point to raise MASQUE's chartered focus on CC in CC. Discussing the MP-DCCP encapsulation framework makes this challenge even stronger and I'm happy to support any attempt to bring this jointly to ICCRG.

From a separation point of view I have no doubt that we can clearly distinguish the MP extension of DCCP from the encapsulation/tunneling stuff. That's already the way how we applied the drafts.

https://tools.ietf.org/html/draft-amend-tsvwg-multipath-dccp only takes care to specify the DCCP extension

while https://tools.ietf.org/html/draft-amend-tsvwg-multipath-framework-mpdccp is intended to be informational and give guidance towards multipath support of non-DCCP traffic using tunneling/encapsulation.

Br

Markus

From: Martin Duke <[email-masked]> 
Sent: Dienstag, 17. November 2020 01:46
To: Amend, Markus <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] MP-DCCP for UDP multipath transport

Hi Markus,

I should note that MASQUE is specifically chartered to provide recommendations for cc-in-cc without designing a new protocol or starting any research projects. The draft probably ought to say *something.*

Header conversion and multipath seem uncontroversial to me, but encapsulation may raise some concerns. Can you tell me how separable the drafts are? Would there be any point in adopting two of them, or is encapsulation the use case that drives all the interest in these changes?

On Mon, Nov 16, 2020 at 11:04 AM <mailto:[email-masked]> wrote:
> Cc: mailto:[email-masked]
> > On 16. Nov 2020, at 18:46, mailto:[email-masked] wrote:
> >
> >> Cc: mailto:[email-masked]
> >>> On 16. Nov 2020, at 17:21, mailto:[email-masked] wrote:
> >>>
> >>>> Cc: mailto:[email-masked]
> >>>>> On 16. Nov 2020, at 16:43, mailto:[email-masked] wrote:
> >>>>>
> >>>>> Hi Michael,
> >>>>>
> >>>>> inline
> >>>>>

That's true

> >>>>>> u==[=[nderhttps://http://datatracker.ietf.org/meeting/109/materials/slides-
> 109-
> >>>> tsvwg-

---

From: Martin Duke <[email-masked]>
Date: Tue, 17 Nov 2020 00:24:14 -0800

Hi Markus,

My separability question is not whether it's technically possible (I'm
convinced it is); it's whether excluding the encapsulation part leaves us
with a use case that's actually worth working on.

On Tue, Nov 17, 2020 at 12:07 AM <[email-masked]> wrote:

> Hi Martin,
>
> Good point to raise MASQUE's chartered focus on CC in CC. Discussing the
> MP-DCCP encapsulation framework makes this challenge even stronger and I'm
> happy to support any attempt to bring this jointly to ICCRG.
>
> From a separation point of view I have no doubt that we can clearly
> distinguish the MP extension of DCCP from the encapsulation/tunneling
> stuff. That's already the way how we applied the drafts.
>
> https://tools.ietf.org/html/draft-amend-tsvwg-multipath-framework-mpdccp
> is intended to be informational and give guidance towards multipath support
> of non-DCCP traffic using tunneling/encapsulation.
>
> Br
>
> Markus
>
> http://datatracker.ietf.org/meeting/109/materials/slides-
> > 109-
> > >>>> tsvwg-

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 08:41:03 +0000

Hi Martin,

My clear motivation was and is the encapsulation use case, however I'm happy to learn about different views from others. The overhead free DCCP-UDP header conversion may inspire people to use in future DCCP in their applications directly, since middle-box issues are prevented - but probably that is a weak assumption under the knowledge that RFC6773 (DCCP-UDP) exists.

In respect to adoption - and sorry here, I had not read your original mail clearly to the end - the question of use cases is really something we should consider.

What would be your conclusion if it turns out, that tunneling is the only/main use case? Specifying  DCCP encapsulation (protocol) separately or within the MP-DCCP?

Br

Markus 

Sent: Dienstag, 17. November 2020 09:24

Hi Markus,

My separability question is not whether it's technically possible (I'm convinced it is); it's whether excluding the encapsulation part leaves us with a use case that's actually worth working on.

On Tue, Nov 17, 2020 at 12:07 AM <mailto:[email-masked]> wrote:
Hi Martin,

Br

Markus

From: Martin Duke <mailto:[email-masked]> 
Sent: Dienstag, 17. November 2020 01:46
To: Amend, Markus <mailto:[email-masked]>
Cc: mailto:[email-masked]; mailto:[email-masked]
Subject: Re: [tsvwg] MP-DCCP for UDP multipath transport

Hi Markus,

On Mon, Nov 16, 2020 at 11:04 AM <mailto:mailto:[email-masked]> wrote:
> From: Michael Tuexen <mailto:mailto:[email-masked]>
> To: Amend, Markus <mailto:mailto:[email-masked]>
> Cc: mailto:mailto:[email-masked]
> > On 16. Nov 2020, at 18:46, mailto:mailto:[email-masked] wrote:
> >
> >> From: Michael Tuexen <mailto:mailto:[email-masked]>
> >> To: Amend, Markus <mailto:mailto:[email-masked]>
> >> Cc: mailto:mailto:[email-masked]
> >>> On 16. Nov 2020, at 17:21, mailto:mailto:[email-masked] wrote:
> >>>
> >>>> From: Michael Tuexen <mailto:mailto:[email-masked]>
> >>>> To: Amend, Markus <mailto:mailto:[email-masked]>
> >>>> Cc: mailto:mailto:[email-masked]
> >>>>> On 16. Nov 2020, at 16:43, mailto:mailto:[email-masked] wrote:
> >>>>>
> >>>>> Hi Michael,
> >>>>>
> >>>>> inline
> >>>>>
> >>>>>> From: Michael Tuexen <mailto:mailto:[email-masked]>
> >>>>>> To: Amend, Markus <mailto:mailto:[email-masked]>
> >>>>>>> On 16. Nov 2020, at 13:52, mailto:mailto:[email-masked] wrote:
> >>>>>>>
> >>>>>>> Dear all,
> >>>>>>>

That's true

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 17 Nov 2020 10:32:28 +0100

> On 17. Nov 2020, at 09:41, [email-masked] wrote:
> 
> Hi Martin,
> 
I have a question related to the U-DCCP format: What is the src/dst port numbers? I guess they are from the UDP
space port number space. But in the DCCP header they are from the DCCP port number space. From which port
number space are they taken?

Best regards
Michael

---

From: Martin Duke <[email-masked]>
Date: Tue, 17 Nov 2020 01:36:49 -0800

On Tue, Nov 17, 2020 at 12:41 AM <[email-masked]> wrote:

>
> What would be your conclusion if it turns out, that tunneling is the
> only/main use case? Specifying  DCCP encapsulation (protocol) separately or
> within the MP-DCCP?
>
> Markus
>

I'm not sure. I does seem to be true that if the other two are valuable on
their own, they have fewer concerns for adoption.

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 09:40:57 +0000

> Sent: Dienstag, 17. November 2020 10:32
> Cc: [email-masked]; [email-masked]
> > My clear motivation was and is the encapsulation use case, however I'm happy
> to learn about different views from others. The overhead free DCCP-UDP header
> conversion may inspire people to use in future DCCP in their applications directly,
> since middle-box issues are prevented - but probably that is a weak assumption
> under the knowledge that RFC6773 (DCCP-UDP) exists.

That's still open and not clarified yet. I remember a comment from Dave when I presented the draft for the first time which went in a similar direction.

I see two possibilities here:

1. Takeover the ones from the original DCCP communication with the risk to interfere with the UDP port space. So probably a showstopper here.

2. To specify a distinct (IANA) (src/)dst port

> Best regards
> Michael
> >
> > In respect to adoption - and sorry here, I had not read your original mail clearly
> to the end - the question of use cases is really something we should consider.
> >
> > What would be your conclusion if it turns out, that tunneling is the only/main
> use case? Specifying  DCCP encapsulation (protocol) separately or within the MP-
> DCCP?
> >
> > Br
> >
> > Markus
> >
> > My separability question is not whether it's technically possible (I'm convinced it
> is); it's whether excluding the encapsulation part leaves us with a use case that's
> actually worth working on.
> >
> > Good point to raise MASQUE's chartered focus on CC in CC. Discussing the MP-
> DCCP encapsulation framework makes this challenge even stronger and I'm
> > From a separation point of view I have no doubt that we can clearly distinguish
> the MP extension of DCCP from the encapsulation/tunneling stuff. That's already
> the way how we applied the drafts.
> >
> > while https://tools.ietf.org/html/draft-amend-tsvwg-multipath-framework-
> mpdccp is intended to be informational and give guidance towards multipath
> support of non-DCCP traffic using tunneling/encapsulation.
> >
> > Br
> >
> > Markus
> >
> recommendations for cc-in-cc without designing a new protocol or starting any
> research projects. The draft probably ought to say *something.*
> >
> drafts are? Would there be any point in adopting two of them, or is encapsulation
> the use case that drives all the interest in these changes?
> >
> > On Mon, Nov 16, 2020 at 11:04 AM
> <mailto:mailto:[email-masked]> wrote:
> >>>>>> From: Michael Tuexen
> <mailto:mailto:[email-masked]>
> >>>>>>>> From: Michael Tuexen
> <mailto:mailto:[email-masked]>
> >>>>>>>>> On 16. Nov 2020, at 13:52,
> mailto:mailto:[email-masked] wrote:
> >>>>>>>>>
> >>>>>>>>> Dear all,
> >>>>>>>>>
> >>>>>>>> 1. You stated that you are using CCID2. This means that you are adding
> to
> >>>>>>>> handling of UDP
> >>>>>> in https://arxiv.org/pdf/1907.04567.pdf based on NADA as application
> CC.
> >>>> More
> >>>>>>>> 2. Considering that you are fine with a tunneling protocol that provides
> a
> >>>>>>>> congestion control,
> >>>>>> in Linux, if the corresponding specification could be finished (Linux
> recently
> >>>> got
> >>>> based approach. Both has from my view value and MP-DCCP is first of all
> only
> >> a
> >> So you are adding something to DCCP? You could do that also as a shim on top
> of
> >> SCTP using the U-bit.
> >>
> >
> defining a modular re-ordering scheme directly as part of the protocol, at least
> the signaling/negotiation for it. So no shim layer is required.
> >
> >
> > In the meantime I also remember another experienced drawback from our
> SCTP prototype. I think it was quit inflexible when new communication paths
> came up during the lifetime of a SCTP session, a scenario often happens in mobile
> use cases. Only with an INIT chunk, communication paths can be negotiated once
> during the handshake. Any new IP addresses/paths, e.g. while changing from
> one Wi-Fi AP to another, requires a re-establishment of the SCTP session.
> >
> >>>>>>>> prototype implementation. This prototype leverages MP-DCCP
> between
> >> a
> >>>>>>>>> Seamless handover for reliability and path/access aggregation for
> high
> >>>>>> speeds
> >>>>>>>>> Especially for 3GPP ATSSS purposes, this very dedicated and simple
> MP-
> >>>> DCCP
> >>>>>>>>> However MP-DCCP is not limited to ATSSS and can be applied end-
> to-
> >> end
> >>>>>> with

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 17 Nov 2020 11:00:32 +0100

> On 17. Nov 2020, at 10:40, [email-masked] wrote:
> 
If you use an IANA reserved UDP port number for DCCP/UDP encapsulation, you can have only one such
endpoint on a host. So how would you support multiple DCCP endpoints using this feature?
This looks like the UDP encapsulation specified in RFC 6951 for SCTP. However, in this solution
UDP and SCTP port numbers both exist...

Best regards
Michael

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 11:04:35 +0000

> Sent: Dienstag, 17. November 2020 11:01
> >> space port number space. But in the DCCP header they are from the DCCP
> port
> >> number space. From which port
> > That's still open and not clarified yet. I remember a comment from Dave when I
> presented the draft for the first time which went in a similar direction.
> >
> > 1. Takeover the ones from the original DCCP communication with the risk to
> interfere with the UDP port space. So probably a showstopper here.
> >

Possible solutions for further discussion:

1. In controlled environments, there is nothing which prevents implementers to use other ports than specified.
2. Fix dst.port for UDP (IANA) and original DCCP dst.port has to be transmitted in the DCCP handshake.
3. Client/Server have to select for each DCCP port an equivalent UDP port and map internally
4. Handshaking procedure for DCPP-UDP header conversion
5. Use DCCP-UDP negotiation procedure to signal support for DCCP-UDP header conversion. If not supported keep encapsulation...

> convinced it
> distinguish
> >>> https://tools.ietf.org/html/draft-amend-tsvwg-multipath-dccp only takes
> care
> >> to specify the DCCP extension
> >>>
> encapsulation
> >>>>>>>>> On 16. Nov 2020, at 16:43,
> adding
> >>>>>>>> recent results are part of our presentation at ICCRG on Friday,
> discussing
> >> a
> >>>>>>>> I agree that this is not specific to the scenario you describe, but a
> broader
> >>>>>> issue.
> provides
> >>>>>> Linux reference implementation of (MP-)DCCP it was much simpler for
> us
> >> to
> >>>>>> multipath extension for DCCP. Using encapsulation on top makes it
> capable
> >> of
> >>>>>> be specified extension, QUIC needs unreliability, and all need a way of
> load
> >>>>>> sharing. But then
> >>>>>>> Please correct me if I'm wrong, but as long as the FORWARD TSN has
> not
> >> be
> >>>>> Yep, I identified we argued from different perspectives. I agree, setting
> the
> >> U-
> top
> >>>>>>>>>>> https://tools.ietf.org/html/draft-amend-iccrg-multipath-
> reordering-
> >> 01
> simple
> >>>>>>>>>> framework provides an alternative over the currently discussed
> MP-
> >>>> QUIC

---

From: Michael Tuexen <[email-masked]>
Date: Tue, 17 Nov 2020 12:36:14 +0100

> On 17. Nov 2020, at 12:04, [email-masked] wrote:
> 
Sure.
> 2. Fix dst.port for UDP (IANA) and original DCCP dst.port has to be transmitted in the DCCP handshake.
Then it will not be zero overhead.
> 3. Client/Server have to select for each DCCP port an equivalent UDP port and map internally
How does the client knows how the server is doing an internal mapping?
> 4. Handshaking procedure for DCPP-UDP header conversion
Is that still zero overhead?
> 5. Use DCCP-UDP negotiation procedure to signal support for DCCP-UDP header conversion. If not supported keep encapsulation...
So you have zero overhead after the handshake. But you have 4 port numbers during the handshake, under
which constraints can you reduce to two. I guess these would be the UDP port numbers...

Best regards
Michael

---

From: <[email-masked]>
Date: Tue, 17 Nov 2020 11:46:27 +0000

> Sent: Dienstag, 17. November 2020 12:36
> >> If you use an IANA reserved UDP port number for DCCP/UDP encapsulation,
> you
> >> can have only one such

You are right for this small piece of DCCP SYN packet, but all following communication is overhead free.

> How does the client knows how the server is doing an internal mapping?

OK, I forgot to mention that this has to be a prerequisite (maybe only a solution for controlled environments), otherwise it requires handshaking.

> Is that still zero overhead?

Not in terms of handshaking, but afterwards then, similar to 5.

> > 5. Use DCCP-UDP negotiation procedure to signal support for DCCP-UDP
> header conversion. If not supported keep encapsulation...

Probably

> 
> Best regards
> Michael
> >
> >
> >> This looks like the UDP encapsulation specified in RFC 6951 for SCTP. However,
> in
> >> this solution
> only/main
> >>>> use case? Specifying  DCCP encapsulation (protocol) separately or within
> the
> >> MP-
> >>>> DCCP?
> >>>>>
> >>>>> Br
> >>>>>
> >>>>> Markus
> >>>>>
> any
> >>>>>>>>> On 16. Nov 2020, at 17:21,
> >>>>>>>>>>>>> as the TSVWG chairs suggested in the IETF 109 session, I
> continue
> >> the
> >>>>>>>>>> discussion
> >>>>>>>>>>>> on the mailinglist.
> >>>>>>>>>>>> Hi Markus,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I have two questions:
> >>>>>>>>>>>>
> >>>>>>>>>>> Yes, and moreover we also implemented BBRv1 into the DCCP
> CCID
> >>>>>>>> framework.
> application
> >>>>>>>>>>> 1. Nested CC is a broader topic beyond MP-DCCP and also valid
> e.g.
> >> for
> >>>>>>>>>>>> what are the benefits of using DCCP instead of using SCTP with
> UDP
> >>>>>>>>>>>> encapsulation? You can
> >>>>>>>>>> prototype, a SCTP based prototype leveraging PR-, PF- and CMT-
> SCTP.
> >>>> The
> all
> >>>>>>>> be fulfilled. So if you send all user messages as unordered messages
> (like
> >>>> on
> >>>>>> UDP
> >>>>>>>> or DCCP),
> >>>>>> immediately to the application layer. That would give support for
> steering
> >> and
> >>>>>> function might be kept and then we step into the FORWARD-TSN trap
> with
> >>>> HoL.
> >>>>>> with a modular re-ordering scheme, which easily let define and use
> various
> >> re-
> least
> >>>>>>>>>>>> sessa-61-dccp-extensions-for-multipath-operation-00, with a
> main
> >>>> focus
> >>>>>> on
> >>>>>>>>>> the
> >>>>>>>>>>>> with the Internet. Due to the MP-DCCP encapsulation
> framework,
> >>>>>> support
> >>>>>>>> is
> >>>>>>>>>> not
> end-
> >>>>>>>>>>>>> So I encourage you to read through the drafts, give feedback
> and
> >>>>>> discuss
> >>>>>>>> the