Subject: Re: [nwcrg] [transport-services] A few questions

From: Michael Welzl <[email-masked]>
Date: Mon, 25 Nov 2013 13:45:28 +0100

[Context: I see that some recipients of this email are not on the transport-services mailing list, so: this is about a use case draft that is being written as we're working towards a "Transport Services" BOF for the London IETF. A page describing this effort and mailing list information is here: https://sites.google.com/site/transportprotocolservices/  ]

On 25. nov. 2013, at 13:08, Marie-Jose Montpetit wrote:

> The use case document is being already revised with some of the comments that were on the list ;-) But let me give you a flavor.

Great, thanks!

> Essentially I looked for services that use transport in a unique way. So right now I have:
> - streaming: with adaptive rates, caching/buffering
> - real time services: TV that adds multicast requirements and VoIP - some of the requirements here are common with rtcweb and I am looking in the reliable multicast work also
> - public security: services are similar but add strong security, reliability and 'always on' - some requirements are shared with multipath TCP

Up to here this sounds very good to me.

> - IoT: I have it for completeness but I am not sure if there is a unique transport there but a unique use of the transport

Not sure either...

> - data centers/storage: because there is work TCP for data centers

I think "there is work on TCP for data centers" by itself isn't a good reason, but I agree with data centers, for another reason. Let me explain:

1) not a good reason: the whole transport services idea is about decoupling the services applications ask for from the choice of TCP and UDP. Data center TCP (DCTCP) is just a way of making TCP work better in data centers, and as such doesn't translate into a different application service or require any different communication with applications than what the normal socket API provides (I also don't think it represents a different performance trade-off or anything like that - if you're in a data center, it just works better than normal TCP). Thus DCTCP wouldn't fit in my opinion.

2) but data centers fit: ... because of other related work - D2TCP (Deadline-aware Datacenter TCP) - now THAT is a great, 100% fitting example to include, as it's about getting a benefit for an application in a way that necessitates changing the API.

> - ICN/CCN: not really transport but, as was mentioned on the list, a different way of signaling requirements
> 
> Most of the above are impacted by wireless errors and I am on the verge of making 'wireless' it's own transport use case (comments?).

See my discussion of DCTCP above. Just ask yourself, does your envisioned use case require a different way of communication between applications and the net (if so: yes, include), or is it just a special environment where we need to generally improve protocols (if so: no)?

> As of now there is a XOR in QUIC (and some other) that can help survive a certain level of errors in the transmission but errors in wireless (both cellular and WIFI) can of course be high. The IRTF has recently chartered a WG on Network Coding to define better error resistant transport using Network Coding but it's very new. And the issue there (and it was my question about commercial APIs) is that there is no Open Source implementations (but I understand some version is coming).

Again, the "communicate-with-application" question. I'm not sure there's a need to explicitly involve applications, but it's quite possible that I'm missing something. But that's the whole value of the use case document, it shows people like me who don't understand these things some example situations where an application really needs a different type of communication with the transport to benefit from e.g. network coding.

> Finally, and I am not sure if I will keep it in the use cases, there is this idea of 'discovery' of the transport services which was my idea at at the start. Some kind of 'transport as a service' idea so that applications could be designed in such a way as to be able to use new transports when available (and not sure of the details).

This sounds like it's not a use case, and it also sounds as if you're envisioning to enrich applications with exactly the type of functionality that we'd like to hide from them?

I mean, the whole point here is to let the application specify a service instead of a protocol, and hide the actual protocol choice from the application. This way, the most suitable protocol or mechanism could be automatically picked out of what is available. The services that the protocols provide need to be exposed to applications in a reasonable way, and this is why we look at the various APIs that middlewares provide. The richer set of services offered by transports beyond just standard TCP and UDP also allows applications to use the transport layer in ways that weren't possible before, and to describe these things we have the use case document. That's my view of the situation, at least, and it seems to match almost everything you describe above, but probably not that 'discovery' bit.

> And I am open to add others use cases of course. 
> 
> So that's almost the document ;-) Except not in the right format...

I'm looking forward to it!

Cheers,
Michael

---

From: Marie-Jose Montpetit <[email-masked]>
Date: Mon, 25 Nov 2013 07:08:52 -0500

Most of the above are impacted by wireless errors and I am on the verge of making 'wireless' it's own transport use case (comments?). As of now there is a XOR in QUIC (and some other) that can help survive a certain level of errors in the transmission but errors in wireless (both cellular and WIFI) can of course be high. The IRTF has recently chartered a WG on Network Coding to define better error resistant transport using Network Coding but it's very new. And the issue there (and it was my question about commercial APIs) is that there is no Open Source implementations (but I understand some version is coming).

Marie-José Montpetit
[email-masked]
[email-masked]

> On Nov 25, 2013, at 4:36, Michael Welzl <[email-masked]> wrote:
> 
> Hi,
> 
> All of that sounds pretty reasonable to me. "Porting existing applications to new APIs" indeed doesn't sound like a good plan. I think we can do two things:
> 1) update middlewares to work better without requiring a change to the API  (which is what we're trying to describe with the API document that Per edits)
> 2) envision some futuristic situations where applications would use a newer, enriched API, to benefit from enhanced network services. This is my interpretation of the use case document (let's see what we really have there, I'm curious to get a first version)
> 
> Cheers,
> Michael
> 
> 
>> On 24. nov. 2013, at 11:46, Martin Sustrik wrote:
>> 
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> 
>>> On 22/11/13 14:15, Marie-Jose Montpetit wrote:
>>> I think that as a start we should look at both network savvy and
>>> non network savvy developpers. And frankly the latter category is
>>> growing faster than the former.
>> 
>> Right. Playing with the APIs for almost 10 years now and seeing how
>> people adopt them, here's couple of thoughts on the topic from
>> adoption persepective (what happens after IETF) as opposed to
>> technical perspective (what happens in IETF):
>> 
>> The concept of networking for 90% of programmers is like this: "Open a
>> connection to an address, send bytes, receive bytes (reliably and
>> in-order), get an error if data cannot be transferred." In short, it's
>> a crude version of what TCP does.
>> 
>> These people are not networking experts and don't aspire to become
>> networking experts either. They are working in different areas and
>> want the networking just go out of their way.
>> 
>> That being said, any change of the API that breaks the above simple
>> mental concept is likely to get ignored. If it is to be accepted we
>> have to bribe the developers by quite a lot in exchange. And by "a
>> lot" I mean that "better QoS" won't cut it.
>> 
>> Thus, what we should thus think about is:
>> 
>> 1. How to make the change to change to developers' mental model
>> minimal. The options here are either modifying the existing TCP-like
>> mental model slightly, or, alternatively, proposing a different model
>> that is as conceptually as simple as the TCP-based one.
>> 
>> 2. What should be offer in exchange? We need to create an immediate
>> gratification for the developers to reward them for the pain of
>> changing their mental model.
>> 
>> One more thought: The practical result of the TCP-based mental model
>> is that in almost all existing applications all the data transferred
>> between points A and B are sent via a single TCP connection. Part of
>> the data may be real-time data that would benefit from being sent via
>> DCCP, some may rightfully require TCP. However, the application mixes
>> it in a single bytestream and there's no simple way to untangle the mess.
>> 
>> Thus, I would said, focusing on porting existing applications to the
>> new API is a lost fight from the beginning. Which in turn means we
>> don't have to focus on backward compatibility as it's likely to be
>> just a wasted effort.
>> 
>> Martin
>> 
>> 
>> 
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.11 (GNU/Linux)
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>> 
>> iQEcBAEBAgAGBQJSkdkYAAoJENTpVjxCNN9YW48IAJmuTsvsvtubwqMaytvgkXuo
>> N6GHxB+FAkX7FGSzXcybyjZNiArTlrgVa0dIHYQG4d8xmhB+o21bi2/LLtP0+0xF
>> 4QijXw1k996CMa7j4yGM3U/OWJYknGv+4Am9ObyxCc5fY0UsxOa9Jlx4lpvKog8l
>> Mi1vcmpLwCbpXIPXpByKir1OcZoFXoG3WcRakkYfFimINV4kCUzjy9Eb1OwXc/xw
>> 6+n+wk4pTpUCyu8YeQkRhXYD0jee+bwKYc1Y4czOX34zp2+qPpTR2Ymx1AyBMz9C
>> gnPsf3tzrSv5HGQ2Y9UrO6+fGVjk/d+moJ4v9I8BG5ZCwvR0oVDGXXxY5pGR/OM=
>> =h6MS
>> -----END PGP SIGNATURE-----
> 
> 
> ---
> Transport Services BOF plan - website: href="https://sites.google.com/site/transportprotocolservices/

---

From: Michael Welzl <[email-masked]>
Date: Mon, 25 Nov 2013 21:01:09 +0100

Hi!

In line:

On 25. nov. 2013, at 19:22, Igor Zhovnirovsky <[email-masked]> wrote:

> Excellent discussion. In the context of your example of DCTCP vs D2TCP below how do you see the API function? Here are two possibilities (may be more):
> 
> 1. there is a list by name of the “transports” available on this platform for an application. Application simply picks one that it was designed to work with.

Absolutely not, this is the opposite of the plan!

> 2. Application requests a connection by specifying a set of requirements. API connects it to the best fit “transport” (falling back to TCP/UDP if nothing fits)

Absolutely, this is exactly the plan!

If there is anything in the description at https://sites.google.com/site/transportprotocolservices/
or the draft charter at https://sites.google.com/site/transportprotocolservices/home/charter-proposal-before-bof
that should be made clearer to avoid such misunderstandings, please let me know.

Cheers,
Michael

---

From: Igor Zhovnirovsky <[email-masked]>
Date: Mon, 25 Nov 2013 18:22:22 +0000

Regards
Igor

On Nov 25, 2013, at 7:45 AM, Michael Welzl <[email-masked]<mailto:[email-masked]>> wrote:

Great, thanks!

Not sure either...

Cheers,
Michael

---

From: Martin Sustrik <[email-masked]>
Date: Tue, 26 Nov 2013 12:00:18 +0100

On 25/11/13 13:45, Michael Welzl wrote:
> 
>> - IoT: I have it for completeness but I am not sure if there is
>> a unique transport there but a unique use of the transport
> 
> Not sure either...

You can have a look at IBM's MQTT protocol which explicitly positions
itself as a protocol for IoT, however, what it boils down to is
collecting data from sensors, which definitely does not encompass all
the IoT use cases.

Martin