Subject: RE: [lemonade] COMPRESS Thoughts

From: <[email-masked]>
Date: Thu, 1 Jun 2006 11:55:48 +0300

Does (2) mean that when the client sends email it cannot compress it?
Thank You. 

Best regards: Zoltán Ördögh
E-mail: zoltan.ordogh at nokia.com
Phone: +358 50 386 0566

-----Original Message-----
From: ext Burger, Eric [mailto:[email-masked]] 
Sent: 31 May, 2006 22:29
To: Enhancements to Internet email to support diverse serviceenivronments
Subject: [lemonade] COMPRESS Thoughts

(1) Going back to COMPRESS in its own document (not CONVERT)

(2) Consensus that compression is directional, i.e., downstream compressed, upstream uncompressed.  This is because IMAP upstream commands may be real short (meaning compression actually adds CPU *and* bits used), while content may be real long.

(3) Basis for COMPRESS document is Arnt's imap-deflate-02 [compress=deflate].  We will call the new document
draft-ietf-lemonade-compress-04 (delete existing compress text)

(4) Protocol allows for client to turn it on or off at will (not per body part, but point in time in session)

(5) review SASL mechanism for handling glare: server is busy streaming down compressed content, and the client says, "stop compressing".  Right now looks like server will continue until finished with IMAP transaction.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 1 Jun 2006 12:30:33 +0200

> body part, but point in time in session)

There's a performance gotcha to this, I realized while changing my 
daughter's nappies at 6am this morning.

The compression library likes to read in big chunks. Everyone likes to 
do I/O in big chunks. Reading network input one byte at a time is such 
a performance drag. If the client tells the server that starting with 
the next command, its commands are uncompressed, the server's 
compression library has to stop reading at the right byte. It can't 
simply read all it can eat. Ditto for responses.

This should fix it:

1. The client cannot send any other commands until the server has sent a 
tagged OK for the 'disable upstream compression' command.

2. The server disables compression after sending the tagged OK for that 
command (including the CRLF).

3. The server cannot send any unsolicited responses after sending that 
tagged OK and before receiving the next command.

The last introduces a new server mode and gets in the way of extending 
IDLE... right now, I think that server mode is more trouble than it's 
worth.

What is the problem if compression stays on once it's turned on? I can't 
recall any relevant problems with STARTTLS, which behaves that way.

Arnt

---

From: Dave Cridland <[email-masked]>
Date: Thu, 01 Jun 2006 13:25:59 +0100

> Does (2) mean that when the client sends email it cannot compress 
No, but the lack of point 6 means that sending email isn't compressed 
- which is why we need to define a similar extension for Submission.

I'll reiterate that you do see compression for data sent from the 
client, I've just found this to be less effective, and I can believe 
that the resources required for compression may outweigh the benefits 
on some clients. (Again, I doubt this is the case for mobile 
handsets, or when encryption is active). Where no compression is 
performed, there is a per-block overhead of 5 octets, and those 
blocks can't be streamed.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Dave Cridland <[email-masked]>
Date: Thu, 01 Jun 2006 13:26:18 +0100

On Thu Jun 01 11:30:33 2006, Arnt Gulbrandsen wrote:
> Eric Burger writes:
>> (4) Protocol allows for client to turn it on or off at will (not 
>> per body part, but point in time in session)
> 
Ah, well while changing my daughter's nappy just now, I decided that 
it's not such a big issue.

> The compression library likes to read in big chunks. Everyone likes 
> to do I/O in big chunks. Reading network input one byte at a time 
> is such a performance drag. If the client tells the server that 
> starting with the next command, its commands are uncompressed, the 
> server's compression library has to stop reading at the right byte. 
> It can't simply read all it can eat. Ditto for responses.
> 
> 
Ah, but in the case of zlib at least, it knows when the deflate 
stream has ended and doesn't consume any further octets, so you can 
extract the data back from the z_stream structure.

> What is the problem if compression stays on once it's turned on? I 
> can't recall any relevant problems with STARTTLS, which behaves 
> that way.

You can turn encryption off within TLS, however. (Or compression). 
There is a minor point that nobody ever does this, however, the 
facility is there if required.

Dave.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 1 Jun 2006 14:45:10 +0200

Dave Cridland writes:
> Ah, but in the case of zlib at least, it knows when the deflate stream 
> has ended and doesn't consume any further octets, so you can extract 
> the data back from the z_stream structure.

Much like the medicine men can extract the pills you shouldn't have 
eaten back from your belly, provided you haven't digested them yet. 
It's just not pleasant. Much better to avoid the whole situation.

Arnt

---

From: Randall Gellens <[email-masked]>
Date: Thu, 1 Jun 2006 11:24:45 -0700

At 12:30 PM +0200 6/1/06, Arnt Gulbrandsen wrote:

>  What is the problem if compression stays on 
> once it's turned on? I can't recall any 
> relevant problems with STARTTLS, which behaves 
> that way.

As I recall, it was Stéphan that has been 
insisting that the client be able to turn off 
compression.

I've been discussing the issue with some handset 
people, and so far everything I've heard is that 
compression is a Good Thing and a client would 
not want to disable it.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
The most likely way for the world to be destroyed, most experts agree,
is by accident.  That's where we come in; we're computer professionals.
We cause accidents.                              --Nathaniel Borenstein

---

From: Ned Freed <[email-masked]>
Date: Thu, 01 Jun 2006 17:16:35 -0700 (PDT)

> recall any relevant problems with STARTTLS, which behaves that way.

The only problem I can recall hearing about leaving STARTTLS on comes up in
news, not email. The situation there is that you may want to use TLS to protect
the authentication exchange, but you really don't want to have it engaged while
transferring gigabytes of data. In theory even this can be dealt with by
negotiating a null cipher and integrity check (perhaps leaving compression on,
come to think of it...), but nobody seemed entirely sure that such
renegotiation mid-session would actually work.

I see no equivalent issue in regard to compression in email in any case.

				Ned

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Jun 2006 15:47:46 -0700

I'd be happy if compression can not be disabled once it has been 
enabled, since that simplifies the extension, and in general simpler 
is better.  Also, from talking to handset people I haven't yet heard 
that there would be a situation where you would want to turn off 
compression.

That still leaves the question of directionality: I assume there are 
still good reasons why clients might want to enable server-to-client 
compression but not client-to-server (since doing so would force them 
to compress their commands)?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
You see, wire telegraph is a kind of very, very long cat.  You
pull his tail in New York and his head is meowing in Los Angeles.
Do you understand this?  And radio operates in exactly the same
way: you send signals here, they receive them there.  The only
difference is that there is no cat.            --Albert Einstein

---

From: Dave Cridland <[email-masked]>
Date: Thu, 08 Jun 2006 23:59:12 +0100

On Thu Jun  8 23:47:46 2006, Randall Gellens wrote:
> enabled, since that simplifies the extension, and in general 
> simpler is better.  Also, from talking to handset people I haven't 
> yet heard that there would be a situation where you would want to 
> turn off compression.
> 
> That still leaves the question of directionality: I assume there 
> are still good reasons why clients might want to enable 
> server-to-client compression but not client-to-server (since doing 
> so would force them to compress their commands)?

As I understand things, for mobile handsets you basically always want 
to compress the upstream traffic, because it saves on radio 
transmission, which is a greater power drain than the CPU.

You may wish to double-check that with handset people - please do, in 
fact.

I wouldn't be entirely happy ruling out other highly resource 
constrained clients wanting to avoid upstream compression, though.

Dave,

---

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Jun 2006 16:34:03 -0700

At 11:59 PM +0100 6/8/06, Dave Cridland wrote:

>  As I understand things, for mobile handsets you basically always 
> want to compress the upstream traffic, because it saves on radio 
>  You may wish to double-check that with handset people - please do, in fact.

I just ran this by some other handset people, and got confirmation 
that, for a handset, compression is always a win.  I was told that 
even receiving radio transmission uses more power than decompressing 
the power (assuming the compression engine gets out of the way for 
already-compressed data).

-------------- Randomly-selected tag: ---------------
You are wise, witty, and wonderful, but you spend too much time reading
this sort of trash.