Subject: Small size of core QUIC library to replace TCP for embedded system

From: Jupiter <[email-masked]>
Date: Tue, 21 Jul 2020 10:30:32 +1000

Hi,

I am looking for a slim backbone core QUIC library to replace TCP stack.

Here are core features currently I am using TCP, I am wondering if
there is a QUIC open source library can be used or to be further
developed to replace following TCP key functionality

- Replace Boost ASIO TCP by QUIC for socket communication between the
device and Cloud services.

- Use (Address)-restricted-cone NAT

- Use TCP TLS

The major issue when I looked at Google and other QUIC source is
because it supports HTTPS, the size is too large, it is not
modularized, Some open source claimed the library can be used by
adding or removing options, the reality is most of the QUIC libraries
are designed based on HTTPS in mind, it is hard to have a clean small
size QUIC core functions above.

I have to admit my knowledge about QUIC is limited, but I am a
programmer, I am keen to participate a modular QUIC open source
development for embedded system, here are options:

- Take an existing QUIC library (C/C++)

- Fork a QUIC Git source to develop a slim C/C++ QUIC for the embedded system.

- Start from scratch based on reference of some QUIC library

Appreciate your advice and suggestions.

Thank you.

Kind regards,

jupiter

--
"A man can fail many times, but he isn't a failure until he begins to
blame somebody else."
-- John Burroughs

---

From: Nick Banks <[email-masked]>
Date: Tue, 21 Jul 2020 00:53:17 +0000

Hello!

Feel free to take a look at MsQuic (https://github.com/microsoft/msquic). It's written in C and is purely a QUIC implementation (no HTTP/3). We've not necessarily targeted it for embedded systems and I can't tell you off-hand how small the library will end up being in your scenario. If you try it out and find any specific problems, feel free to open issues and we can discuss fixing them.

Thanks,
- Nick

-----Original Message-----
From: QUIC <[email-masked]> On Behalf Of Jupiter
Sent: Monday, July 20, 2020 5:31 PM
To: quic <[email-masked]>
Subject: Small size of core QUIC library to replace TCP for embedded system

Hi,

Here are core features currently I am using TCP, I am wondering if there is a QUIC open source library can be used or to be further developed to replace following TCP key functionality

- Replace Boost ASIO TCP by QUIC for socket communication between the device and Cloud services.

- Use TCP TLS

The major issue when I looked at Google and other QUIC source is because it supports HTTPS, the size is too large, it is not modularized, Some open source claimed the library can be used by adding or removing options, the reality is most of the QUIC libraries are designed based on HTTPS in mind, it is hard to have a clean small size QUIC core functions above.

I have to admit my knowledge about QUIC is limited, but I am a programmer, I am keen to participate a modular QUIC open source development for embedded system, here are options:

Thank you.

Kind regards,

jupiter

--
"A man can fail many times, but he isn't a failure until he begins to blame somebody else."
-- John Burroughs

---

From: Jana Iyengar <[email-masked]>
Date: Mon, 20 Jul 2020 20:11:23 -0700

Hi Jupiter,

You might consider the various implementations listed at
https://github.com/quicwg/base-drafts/wiki/Implementations. All the
implementers are on this list, so you should be able to reach them here.

- jana

On Mon, Jul 20, 2020 at 5:53 PM Nick Banks <nibanks=
[email-masked]> wrote:

> Hello!
>
> Feel free to take a look at MsQuic (https://github.com/microsoft/msquic).
> It's written in C and is purely a QUIC implementation (no HTTP/3). We've
> not necessarily targeted it for embedded systems and I can't tell you
> off-hand how small the library will end up being in your scenario. If you
> try it out and find any specific problems, feel free to open issues and we
> can discuss fixing them.
>
> Thanks,
> - Nick
>
> Here are core features currently I am using TCP, I am wondering if there
> is a QUIC open source library can be used or to be further developed to
> replace following TCP key functionality
>
> The major issue when I looked at Google and other QUIC source is because
> it supports HTTPS, the size is too large, it is not modularized, Some open
> source claimed the library can be used by adding or removing options, the
> reality is most of the QUIC libraries are designed based on HTTPS in mind,
> it is hard to have a clean small size QUIC core functions above.
>
> I have to admit my knowledge about QUIC is limited, but I am a programmer,
> I am keen to participate a modular QUIC open source development for
> embedded system, here are options:
>

---

From: Lars Eggert <[email-masked]>
Date: Tue, 21 Jul 2020 09:39:58 +0300

Hi,

> I am looking for a slim backbone core QUIC library to replace TCP stack.

you don't really say what type of device you are targeting, so it's a bit difficult to respond in detail.

But I did port my quant QUIC stack (https://github.com/NTAP/quant) to Particle DeviceOS and RIOT, and ran some client-only measurements on a Particle Argon and an ESP32. There's a writeup in https://eggert.org/papers/2020-ndss-quic-iot.pdf, slides in https://github.com/t2trg/2020-ietf107/blob/master/slides/22-towards-securing-the-internet-of-things-with-quic.pdf. (There was also a recording of the T2TRG talk, but it doesn't seem to be uploaded to GitHub yet.)

It should be reasonably straightforward to add a packet I/O backend for other IoT platforms to warpcore, (https://github.com/NTAP/warpcore) which is quant's packet I/O library. I had been considering Zephyr support, but ran out of time.

Note: quant is a research effort and not meant for production.

Lars

---

From: Jupiter <[email-masked]>
Date: Tue, 21 Jul 2020 18:17:36 +1000

Thanks Lars and all responses, I am very grateful for all of your kind
comments and advice.

> difficult to respond in detail.

Sorry for neglecting that part, I am running Linux on iMX6ULZ using
all sorts of open source libraries in C and C++.

> But I did port my quant QUIC stack (https://github.com/NTAP/quant) to
> Particle DeviceOS and RIOT, and ran some client-only measurements on a
> Particle Argon and an ESP32. There's a writeup in
> https://eggert.org/papers/2020-ndss-quic-iot.pdf, slides in
> https://github.com/t2trg/2020-ietf107/blob/master/slides/22-towards-securing-the-internet-of-things-with-quic.pdf.
> (There was also a recording of the T2TRG talk, but it doesn't seem to be
> uploaded to GitHub yet.)

Thanks for the links, I have to admit, my QUIC knowledge is limited, I
thought QUIC should be a user space program, anything talking to low
layers through Linux kernel API, seems you have to alter Ethernet RX /
TX for QUIC, does that mean you have to alter WiFi or other media
communications as well? My WiFi is currently supported by Linux open
source mwifiex.

> It should be reasonably straightforward to add a packet I/O backend for
> other IoT platforms to warpcore, (https://github.com/NTAP/warpcore) which is
> quant's packet I/O library. I had been considering Zephyr support, but ran
> out of time.

Looks very good, I suppose I should be able to run it on an Ubuntu 18
laptop, right?

> Note: quant is a research effort and not meant for production.

Understood, let me see if I can make some contributions for the project.

Thank you.

Kind regards,

jupiter

---

From: Lars Eggert <[email-masked]>
Date: Tue, 21 Jul 2020 11:26:29 +0300

Hi,

On 2020-7-21, at 11:17, Jupiter <[email-masked]> wrote:
> I am running Linux on iMX6ULZ using

if your hardware can run Linux, most of the current open source stacks (see https://github.com/quicwg/base-drafts/wiki/Implementations) should be small enough to work. But I wouldn't call it an embedded system then...

> layers through Linux kernel API,

That is how most implementations work, correct.

But constrained-resource embedded devices at the low end often can't run Linux or another full-fledged OS. Many hence link together an application with only the required bits of a library OS (RIOT, Zephyr, FreeRTOS & derivatives) and run that statically on bare metal.

> seems you have to alter Ethernet RX /

You do not need to do this for QUIC, only when you want to run on embedded boards that don't have a capable-enough Socket API interface.

> Looks very good, I suppose I should be able to run it on an Ubuntu 18

It will run on Linux without changes (my public test server runs Ubuntu).

Lars

---

From: Jupiter <[email-masked]>
Date: Tue, 21 Jul 2020 20:56:22 +1000

Hi Lars,

On 7/21/20, Lars Eggert <[email-masked]> wrote:
> if your hardware can run Linux, most of the current open source stacks (see
> https://github.com/quicwg/base-drafts/wiki/Implementations) should be small
> enough to work. But I wouldn't call it an embedded system then...

Sorry for not being clear, I should say Linux embedded system at the
beginning. Although I would say the time for embedded systems with 8M
Flash and 64KRAM has been long gone :-), I still have to manage the
resources for meaning full network stacks and applications.

> But constrained-resource embedded devices at the low end often can't run
> Linux or another full-fledged OS. Many hence link together an application
> with only the required bits of a library OS (RIOT, Zephyr, FreeRTOS &
> derivatives) and run that statically on bare metal.

With all due respect, unless you are working on IoT for tiny devices
cannot run Linux, it is 2020, hardware is cheaper, software
development is expensive, Linux and open source become dominant force
in contemporary embedded system, your Android mobile phone is an
embedded system running Linux :-), Sorry let's stop it, we are too far
off topic.

> You do not need to do this for QUIC, only when you want to run on embedded
> boards that don't have a capable-enough Socket API interface.

Good to know, fair to say your git source is for RTOS statically on
bare metal, I was there a long time before :-).

> It will run on Linux without changes (my public test server runs Ubuntu).

I'll try it, thanks Lars.

jupiter

---

From: Paul Vixie <[email-masked]>
Date: Tue, 21 Jul 2020 17:02:54 +0000

On Tuesday, 21 July 2020 10:56:22 UTC Jupiter wrote:
> > ...
> > https://github.com/quicwg/base-drafts/wiki/Implementations)
> > ...
> 

i realize that this has been declared off-topic, but you've raised a question 
that your /quicwg/ URI above does not answer.

for privilege separation reasons, sockets have been kernel objects. this has 
facilitated logging and filtering for regulatory, policy, ad-blocking, malware 
filtering, and so on. user mode code has not historically been able to form a 
connection without a "connect" system call, where policy could be enforced by 
the kernel. as an example, "raw" sockets necessary to implement user-mode TCP 
have historically been restricted to processes running with elevated 
privilege.

in some environments this will be an invariant, and UDP sockets will be made 
to require elevated privileges, precisely so as to give the kernel a chance to 
log and filter connection attempts according to local policy. in that 
environment, a kernel-resident QUIC implementation will be needed, so that 
something like a "connect" system call can still exist. i do not expect this 
to save memory, because shared libraries are equal in size to kernel modules. 
but as a place to put an ad-blocker or other edge-device connection policy, a 
kernel-resident QUIC implementation is going to be required in some places.

my question is, has any QUIC implementor been thinking along these lines? i 
believe that the parameters to a QUIC "connect" syscall would be larger and 
more numerous than for a TCP or SCTP or UDP "connect" syscall, for example.

-- 
Paul

---

From: Lars Eggert <[email-masked]>
Date: Wed, 22 Jul 2020 09:10:44 +0300

Hi,

> bare metal, I was there a long time before :-).

actually, it's not. I *ported* quant to embedded OSs, which was pretty doable since the packet I/O stuff was already factored out into warpcore. But quant as a QUIC stack runs fine on POSIX.

(Warpcore is a kernel bypass IP/UDP stack with a focus on datacenter usage, not IoT. But it has a POSIX backend as well as kernel-bypass support.)

Lars

---

From: Lars Eggert <[email-masked]>
Date: Wed, 22 Jul 2020 09:17:33 +0300

Hi,

> more numerous than for a TCP or SCTP or UDP "connect" syscall, for example.

most QUIC implementations have an API that is at a high level pretty similar to the socket API, but different in all the details. And they are all different from each other.

Since you asked about a "connect" equivalent, here is what some of the C stacks offer:

quant: https://github.com/NTAP/quant/blob/master/lib/include/quant/quant.h#L110-L118

msquic: https://github.com/microsoft/msquic/blob/master/src/inc/msquic.h#L745-L757

quicly: https://github.com/h2o/quicly/blob/master/include/quicly.h#L892-L900

Pretty different.

Lars