Subject: Re: [Int-area] IP Parcels improves performance for end systems

From: "Templin (US), Fred L" <[email-masked]>
Date: Tue, 22 Mar 2022 17:38:04 +0000

Tom, see below:

> -----Original Message-----
> From: Tom Herbert [mailto:[email-masked]]
> Sent: Tuesday, March 22, 2022 10:00 AM
> To: Templin (US), Fred L <[email-masked]>
> Cc: Eggert, Lars <[email-masked]>; [email-masked]
> Subject: Re: [Int-area] IP Parcels improves performance for end systems
> 
> On Tue, Mar 22, 2022 at 7:42 AM Templin (US), Fred L
> <[email-masked]> wrote:
> >
> > Lars, I did a poor job of answering your question. One of the most important aspects of
> >
> > IP Parcels in relation to TSO and GSO/GRO is that transports get to use a full 4MB buffer
> >
> > instead of the 64KB limit in current practices. This is possible due to the IP Parcel jumbo
> >
> > payload option encapsulation which provides a 32-bit length field instead of just a 16-bit.
> >
> > By allowing the transport to present the IP layer with a buffer of up to 4MB, it reduces
> >
> > the overhead, minimizes system calls and interrupts, etc.
> >
> >
> >
> > So, yes, IP Parcels is very much about improving the performance for end systems in
> >
> > comparison with current practice (GSO/GRO and TSO).
> 
> Hi Fred,
> 
> The nice thing about TSO/GSO/GRO is that they don't require any
> changes to the protocol as just implementation techniques, also
> they're one sided opitmizations meaning for instance that TSO can be
> used at the sender without requiring GRO to be used at the receiver.
> My understanding is that IP parcels requires new protocol that would
> need to be implemented on both endpoints and possibly in some routers.

It is not entirely true that the protocol needs to be implemented on both
endpoints . Sources that send IP Parcels send them into a Parcel-capable path
which ends at either the final destination or a router for which the next hop is
not Parcel-capable. If the Parcel-capable path extends all the way to the final
destination, then the Parcel is delivered to the destination which knows how
to deal with it. If the Parcel-capable path ends at a router somewhere in the
middle, the router opens the Parcel and sends each enclosed segment as an
independent IP packet. The final destination is then free to apply GRO to the
incoming IP packets even if it does not understand Parcels.

IP Parcels is about efficient shipping and handling just like the major online
retailer service model I described during the talk. The goal is to deliver the
fewest and largest possible parcels to the final destination rather than
delivering lots of small IP packets. It is good for the network and good for
the end systems both. If this were not true, then Amazon would send the
consumer 50 small boxes with 1 item each instead of 1 larger box with all
50 items inside. And, we all know what they would choose to do.

> Do you have data that shows the benefits of IP Parcels in light of
> these requirements?

I have data that shows that GSO/GRO is good for packaging sizes up to 64KB
even if the enclosed segments will require IP fragmentation upon transmission.
The data implies that even larger packaging sizes (up to a maximum of 4MB)
would be better still.

Fred
 
> Thanks,
> Tom
> 
> >
> >
> >
> > Thanks - Fred
> >
> > _______________________________________________
> > Int-area mailing list
> > [email-masked]
> > https://www.ietf.org/mailman/listinfo/int-area

---

From: Tom Herbert <[email-masked]>
Date: Wed, 23 Mar 2022 06:18:43 -0700

> The data implies that even larger packaging sizes (up to a maximum of 4MB)

Fred,

You seem to be only looking at the problem from a per packet cost
point of view. There is also per byte cost, particularly in the
computation of the TCP/UDP checksum. The cost is hidden in modern
implementations by checksum offload, and for segmentation offload we
have methods to preserve the utility of checksum offload. IP parcels
will have to also leverage checksum offload, because if the checksum
is not offloaded then the cost of computing the payload checksum in
CPU would dwarf any benefits we'd get by using segments larger than
64K.

Tom

---

From: "Templin (US), Fred L" <[email-masked]>
Date: Wed, 23 Mar 2022 13:54:24 +0000

Hi Tom,

> Sent: Wednesday, March 23, 2022 6:19 AM
> Cc: Eggert, Lars <[email-masked]>; [email-masked]; [email-masked]

There is plenty of opportunity to apply hardware checksum offload since
the structure of a Parcel will be very standard. My experiments have been
with a protocol called LTP which is layered over UDP/IP as some other
upper layer protocols are. LTP includes a segment-by-segment checksum
that is used at its level in the absence of lower layer integrity checks, so
for larger Parcels LTP would use that and turn off UDP checksums
altogether. As far as I am aware, there are currently no hardware
checksum offload implementations available for calculating the
LTP checksums.

Speaking of standard, AFAICT GSO/GRO are doing something very
non-standard. GSO seems to be coding the IP ID field in the IPv4
headers of packets with DF=1 which goes against RFC 6864. When
DF=1, GSO cannot simply claim the IP ID and code it as if there were
some sort of protocol. Or, if it does, there would be no way to
standardize it.

Fred

---

From: Tom Herbert <[email-masked]>
Date: Wed, 23 Mar 2022 12:37:24 -0400

On Wed, Mar 23, 2022, 9:54 AM Templin (US), Fred L <
[email-masked]> wrote:

> altogether.

You can't turn it off UDP checksums for IPv6 (except for narrow case of
encapsulation).

As far as I am aware, there are currently no hardware
> checksum offload implementations available for calculating the

If it's a standard per packet Internet checksum then a lot of HW could do
it. If it's something like CRC32 then probably not.

LTP is a nice experiment, but I'm more interested as to the interaction
between IP parcels and TCP or QUIC.

> some sort of protocol. Or, if it does, there would be no way to

There was quite a bit of work and discussion on this in Linux. I believe
the deviation from the standard was motivated by some deployed devices
required the IPID be set on receive, and setting IPID with DF equals to 1
is thought to be innocuous. You may want to look at Alex Duyck's papers on
UDP GSO, he wrote a lot of code in this area.

Tom