Subject: RE: IPCOMP and IPSEC

From: Avram Shacham <[email-masked]>
Date: Fri, 29 May 1998 10:36:10 -0700

Steve,

At 11:56 AM 5/29/98 +0100, Stephen Waters wrote:

>The IP-layer compression (IPPCP) calls
>for zero-history compression - for which the performance can be bad.

Tests show different results.

Sure, an IP datagram with payload that has been previously compressed tends
not to compress any further, regardless of the protocol-layer the
compression is applied in. In other words, compressed data will not
compress further in V42.bis, PPP, or IPComp.  Therefore the IPComp protocol
recommends implementation of an adaptive algorithm to avoid the performance
hit.

Using the Calgary files benchmark with LZS algorithm, the compression ratio
of IP datagrams with MTU of 1500 octets was 1.82:1, not that far from the
max the algorithm can offer 2.2:1.  Stateless compression is a must in IP,
so the overall performance gain is pretty significant.  And, stateless
compression has its implementation advantages.

>When we did LZS zero-history support for X.25 links, we used a minimum
>packet size of 500bytes, i.e. we would not even try to compress the
>packet if it was smaller than that. 

My measurements of LZS with MTU of 576 show a compression ratio of 1.5:1.
MTU of 256 gave a similar ratio, but 41% of the packets failed to compress,
so overall performance got a hit.

The IPComp protocol does recognize that small datagrams are likely to
expand as a result of compression.   Therefore, the protocol suggests that
a numeric threshold should be applied before compression, where IP
datagrams of size smaller than the threshold are sent in the original form
without attempting compression.  The numeric threshold is implementation
dependent.  The algorithms drafts (DELATE, LZS) suggest using a threshold
of ~90 bytes.  My implementation uses the value of 256.

>On my office link (ISDN/PPP/LZS with history),  I average 2:1
>compression.   I suspect this would reduce noticeably for zero-history.

You may be surprised, and - again - for the better.  On my home-office link
(IPComp/ISDN) I got 1.82:1 compression ratio - with or without ESP encryption.

Regards,
avram

---

From: Avram Shacham <[email-masked]>
Date: Fri, 29 May 1998 13:37:01 -0700

Eric,

At 04:20 PM 5/29/98 -0400, Eric Dean wrote:

>Streaming a contiguous file through a compression device is not 
>indicative of real Internet traffic.  Packets of various application are 
>interleaved within flows.  The Calgary files may be a good benchmark for 
>comparing different compression algorithms in a stateful environment; 
                                                 ^^^^^^^^ how comes?

>however, they do not represent the stateless environment that the 
>Internet represents.

In my tests, I ftp-ed and http-ed random collection of files, with
identical results, so it seems that the Calgary Files are a pretty good
indication for non-pre-compressed files.

avram

---

From: Eric Dean <[email-masked]>
Date: Fri, 29 May 1998 16:20:43 -0400 (EDT)

-eric

---

From: Eric Dean <[email-masked]>
Date: Sat, 30 May 1998 17:09:50 -0400 (EDT)

> >comparing different compression algorithms in a stateful environment; 

Looking at the queue state of a typical Internet router, there is next to 
no correlation of any packets within the queue.  Therefore, maintaining a 
state (or history) is not of significant value because the patterns are 
random.  For instance, I rarely have two packets belonging to the same 
flow (or session) in my queue at the same time.

Now, if one were to FTP some files, one at a time, and have no other 
concurrent activity, then the queue serving the local segment would 
contain only packets from the single session.  These packets would be 
highly correlated and therefore should compress relatively well (assuming 
that they are not already compressed).

>From a packet compression perspective, the Internet is a stateless 
environment.  Now if one were to build VPN's with IPSEC, then one might 
achieve better compression ratios.

-eric