Subject: Submit Direction (was RE: [lemonade] COMPRESS Thoughts)

From: "Burger, Eric" <[email-masked]>
Date: Tue, 13 Jun 2006 05:00:55 -0400

Mea Culpa: I used poor language when attempting to describe the
asymmetrical nature of compression.  Re-reading what I wrote, and the
subsequent thread, it is apparent that what I wrote made it sound like
we were ONLY going to have compression in the Server->Client direction,
and we would NOT have compression in the Client->Server direction.

Thing 1: That was not the consensus.  The consensus was there could be
situations where you would want compression all the time in both
directions (the typical case); there could be situations where you would
want compression only Server->Client but not Client->Server; for that
matter, there theoretically could be situations where you would want
compression only Client->Server (although I cannot think of any).  The
point I was trying to make is that we needed to handle compression in
one direction, AS WELL AS compression in both directions.

Thing 2: The entire discussion revolved around IMAP.  As has been
pointed out on the list, compression would be an incredibly useful
feature for SUBMIT.

Thing 3: Compression on SUBMIT does not depend on compression for IMAP.

Thing 4: Lemonade does not strictly depend on compression to get
published.

Thing 5: I don't want to expand our work before we finish our committed
work.

Given these Things, I would offer we work out Compression for IMAP
first.  We need data collection (what resources get saved, what
resources get spent, is CPU really that much more expensive than Radio
[I thought not :->], etc.).  Once we finish Compression for IMAP, doing
a Compression for SUBMIT should be a trivial exercise for the reader.

OK?

On Wed May 31 20:29:26 2006, Burger, Eric wrote:
[snip]
> (2) Consensus that compression is directional, i.e., downstream
> compressed, upstream uncompressed.  This is because IMAP upstream
> commands may be real short (meaning compression actually adds CPU 
> *and* bits used), while content may be real long.
[snip]

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Ted Hardie <[email-masked]>
Date: Tue, 13 Jun 2006 10:53:23 -0700

At 5:00 AM -0400 6/13/06, Burger, Eric wrote:
>.

Speaking as a technical contributor, I think it is better to consider
at least one aspect of the problem together:  the compression and
decompression costs.  There are, for example,  algorithms like Deflate
that use a single-pass deflation but require two passes for
compression.  If we don't take into account now the likelihood that
there will be compression in the terminal, we may end up preferring
methods or algorithms that have different trade-offs when
the terminal is doing the compression.

			regards,
				Ted Hardie

---

From: Randall Gellens <[email-masked]>
Date: Tue, 13 Jun 2006 12:12:34 -0700

>  one direction, AS WELL AS compression in both directions.

True, but subsequent discussion has led me to doubt this requirement. 
That is, I think we need to consider the additional complexity and 
any other costs of allowing for compression in only one direction 
against its utility.  If a client, especially on a mobile device, 
which is capable of compression (and decompression) would always want 
it enabled, then there may not be any use in adding additional 
complexity and commands to negotiate compression in each direction 
separately.

>  pointed out on the list, compression would be an incredibly useful

During the discussion I think there was mention of the benefit for 
submission.  My recollection is that it was mentioned as one of the 
main benefits of client to server compression (the other being the 
fact that many IMAP clients compose command sequences such that only 
a few items differ between them).

>  Thing 3: Compression on SUBMIT does not depend on compression for IMAP.

Depends what is meant by "depend" -- I think the mechanisms and 
negotiation should be as similar as possible.

>  Thing 4: Lemonade does not strictly depend on compression to get

You mean that we can publish profile-bis and the other work that is 
referenced by it before compression is finished.  I'd agree with 
that.  We can always do another update of profile to include 
compression.

>  Thing 5: I don't want to expand our work before we finish our committed

I'd agree, but I thought compression was in our work, and compression 
includes message submission as well as retrieval.

>  a Compression for SUBMIT should be a trivial exercise for the reader.

I'd rather consider both submission and retrieval together.  If that 
means we publish profile-bis before publishing compression, that 
would be OK.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Who tells the stories of a culture really governs human behavior.  It
used to be the parent, the school, the church, the community.  Now it's
a handful of global conglomerates that have nothing to tell, but a
great deal to sell.
    --George Gerbner, dean of Annenberg School of Communication

---

From: Dave Cridland <[email-masked]>
Date: Tue, 13 Jun 2006 20:38:17 +0100

On Tue Jun 13 20:12:34 2006, Randall Gellens wrote:
> True, but subsequent discussion has led me to doubt this 
> requirement. That is, I think we need to consider the additional 
> complexity and any other costs of allowing for compression in only 
> one direction against its utility.  If a client, especially on a 
> mobile device, which is capable of compression (and decompression) 
> would always want it enabled, then there may not be any use in 
> adding additional complexity and commands to negotiate compression 
> in each direction separately.
> 
> 
Something Ted said made me think - not only might there be 
(non-mobile) cases where compression is too expensive for the device, 
but there might also be cases where a different algorithm could be 
used in each direction.

> fact that many IMAP clients compose command sequences such that 
> only a few items differ between them).
> 
> 
FWIW, I've implemented this, and will happily provide xml2rfc or text 
defining it for ESMTP and ACAP, too.

> negotiation should be as similar as possible.
I think they're so astoundingly simple to define for IMAP, ACAP and 
ESMTP all on one document that we might as well. (I know ACAP isn't 
required for Lemonade, it's just sufficiently simple to define all 
three in one place. POP3 could also be added sanely).

> that.  We can always do another update of profile to include 
We have a generally good specification for compression in IMAP. While 
I agree that we wouldn't want to hold up Profile bis "just" for 
compression, I also think it's worth getting compression done, and I 
believe compression to be pretty much ready.

As I see compression, bar some formality of reapplying the whole 
thing to ESMTP etc, the only remaining open issues are:

a) if, and how, we turn compression off on a running connection,

b) if we allow assymetric compression algorithms (including no 
compression),

c) whether we can apply compression before SASL (yes please), and if 
so, whether we can then retrospectively apply a SASL 
encryption/integrity layer. (Up to server implementors, I think it's 
easy, I may be deluded - this requires insertion of a layer into the 
stack at runtime, I think it can be specified safely and negotiated 
by both ends at runtime).

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Ted Hardie <[email-masked]>
Date: Tue, 13 Jun 2006 12:45:18 -0700

At 8:38 PM +0100 6/13/06, Dave Cridland wrote:
>O
>>
>Something Ted said made me think - not only might there be (non-mobile) cases where compression is too expensive for the device, but there might also be cases where a different algorithm could be used in each direction.

That's certainly true, and it may mean we don't need to be too concerned about
picking a single compression algorithm for both directions.  If the numbers support
that, I would certainly be fine with that result.  I just think it is more sensible
to consider them at the same, since that allows us to take into account all
of the variable (compression and decompression efficiency, plus total number
of methods for which support is needed).

Just my opinion,
				Ted Hardie

---

From: Randall Gellens <[email-masked]>
Date: Tue, 13 Jun 2006 15:32:10 -0700

At 8:38 PM +0100 6/13/06, Dave Cridland wrote:

>  b) if we allow assymetric compression algorithms (including no compression),
>
>  c) whether we can apply compression before SASL (yes please), and 
> if so, whether we can then retrospectively apply a SASL 
> encryption/integrity layer. (Up to server implementors, I think 
> it's easy, I may be deluded - this requires insertion of a layer 
> into the stack at runtime, I think it can be specified safely and 
> negotiated by both ends at runtime).

This last one scares me.  I think we need to be clear on how 
compression, SASL, and TLS work together and in what order they are 
negotiated.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Green light in A.M. for new projects.  Red light in P.M. for traffic
tickets.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 13 Jun 2006 23:39:28 +0100

> compression, SASL, and TLS work together and in what order they are 

Well, here's my view:

Compressing before SASL yields a marked saving for the SASL 
exchanges, especially for exchanges like DIGEST-MD5 where there's 
significant chatter. So I want this facility.

I don't think it's a big deal to insert an encryption layer 
underneath a running compression layer, as long as the point where 
that happens is clearly specified. I think it's probably a layer 
violating hack, but equally I don't think it'd cause anyone any 
problems.

However, if the consensus is that it's too scary, I'd be content with 
specifying that SASL MUST NOT negotiate an encryption or integrity 
layer if compression has already been negotiated.

Dave.

---

From: Tony Finch <[email-masked]>
Date: Wed, 14 Jun 2006 00:07:23 +0100

On Tue, 13 Jun 2006, Randall Gellens wrote:
> >  c) whether we can apply compression before SASL (yes please), and if so,
> > whether we can then retrospectively apply a SASL encryption/integrity layer.
> > (Up to server implementors, I think it's easy, I may be deluded - this
> > requires insertion of a layer into the stack at runtime, I think it can be
> > specified safely and negotiated by both ends at runtime).
>
> This last one scares me.  I think we need to be clear on how compression,
> SASL, and TLS work together and in what order they are negotiated.

It's probably worth looking at http://www.jabber.org/jeps/jep-0170.html
since XMPP already has TLS, SASL, and compression layers, and documented
layer ordering. If lemonade decides differently the JEP may be updated to
follow suit.

Given that you know fairly early on whether SASL is going to result in an
encryption layer, based on the mechanism you choose and perhaps
mechanism-specific options, it might be worth varying the recommended
order accordingly.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BAILEY: WEST OR NORTHWEST 5 OR 6, VEERING NORTH 4 OR 5, THEN BACKING
SOUTHWEST 5 OR 6 LATER. OCCASIONAL RAIN. MODERATE OR GOOD.