Subject: Re: compressed content-transfer-encoding?
From: Keith Moore <[email-masked]>
To: Dave Crocker <[email-masked]>
Cc: [email-masked], "Tim Kehres" <[email-masked]>, [email-masked]
Date: Sat, 31 Jul 1999 13:39:47 -0400

> We really do need to remember just how limited bandwidth (in particular) 
> and "Internet performance" are in much of the Internet.  Added to that is 
> that much of the user community pays a usage fee for connection time.

sure, but compressing modems *are* widely deployed.  if the "connection"
uses a compressing modem, any additional savings in "connection time"
resulting from smtp or mime compression is truly marginal.

you'll get more connection time savings from pipelining smtp than you 
will from compressing the data.

Keith

From: Keith Moore <[email-masked]>
To: "Larry Masinter" <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Sat, 31 Jul 1999 13:36:40 -0400
Sender: [email-masked]

> > > Even though application/zip is lame, other mechanisms might be
> > > worse. Maybe we should look harder at providing what would be
> > > necessary to make 'application/zip' actually work, e.g., some
> > > top-level indication of what's actually in the package?
> > 
> > this has the same deployment barrier as adding a new 
> > content-transfer-encoding - in either case the mime mail
> > reader needs to know what to do with the extra parameter
> > or the new c-t-e.
> 
> I'm not sure this is true. Unaware MIME mailers will just
> see they have application/zip, that they either recognize
> or not, and users of existing MIME mail programs have simple 
> way of configuring their mail reader to at do something
> sensible with it.

not clear.  it's one thing to add an ordinary content-type to an
existing mail reader, quite another to add a content-type that
says "decode this body part and then dispatch to the appropriate
content-type handler for its contents"  especially if the 
contents can contain multiple files, or multiparts, or signed
objects.  etc.

> Adding a new content-transfer-encoding has a more serious
> deployment problem, because most deployed systems don't have
> any kind of extensibility built in for CTE.

yes, but my point is that basically you have to upgrade the MUA 
anyway to make this work well.  might as well do it right.

> Mailing around zip files is common practice. Many people are
> used to dealing with getting a zip file.  Leveraging this
> just means making the common practice easier to accomplish
> for senders, and less awkward for receivers; it's a product
> enhancement that mail client vendors could add today.

it's by no means a common practice for everyone - just for some 
users of certain platforms.  even the most common platform on 
which zip is used doesn't ship with zip support.  so no, in 
general, people are not used to dealing with getting a zip file.

so what you are proposing to do is to clutter up the MIME
architecture and degrade the recipient's user interface
just so a minority of users who already use zip don't have
to upgrade immedately.  in the long run I don't think it's
worth it. 

Keith

Date: Sat, 31 Jul 1999 10:23:13 -0700
To: Keith Moore <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Cc: Jacob Palme <[email-masked]>, [email-masked]
Sender: [email-masked]

At 07:28 AM 7/29/99 , Keith Moore wrote:
> > We already have a standard for sending compressed data in e-mail.
> > We have an IANA registered content-type application/zip.
>
>application/zip is not a standard.

Just to beat this one into the ground:

It's also not a compression mechanism.

It does "bagging" of independent obkects, but does not compress the bits.

d/

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dave Crocker                                         Tel: +1 408 246 8253
Brandenburg Consulting                               Fax: +1 408 273 6464
675 Spruce Drive                             <http://www.brandenburg.com>
Sunnyvale, CA 94086 USA                 <mailto:[email-masked]>

Date: Sat, 31 Jul 1999 10:26:06 -0700
To: [email-masked]
From: Dave Crocker <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Cc: "Tim Kehres" <[email-masked]>, [email-masked]
Sender: [email-masked]

At 08:16 AM 7/29/99 , [email-masked] wrote:
>I'm also distressed at this cavalier "with the cost of..." trend of late.
>
>Some places run on tight budgets or have other constraints.  I turned

As in, most of the world.

We really do need to remember just how limited bandwidth (in particular) 
and "Internet performance" are in much of the Internet.  Added to that is 
that much of the user community pays a usage fee for connection time.

And it will be a very long time before things improve.

d/

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dave Crocker                                         Tel: +1 408 246 8253
Brandenburg Consulting                               Fax: +1 408 273 6464
675 Spruce Drive                             <http://www.brandenburg.com>
Sunnyvale, CA 94086 USA                 <mailto:[email-masked]>

Date: Sat, 31 Jul 1999 10:05:49 -0700
To: "Larry Masinter" <[email-masked]>, <[email-masked]>
From: Paul Hoffman / IMC <[email-masked]>
Subject: RE: compressed content-transfer-encoding? 
Cc: <[email-masked]>
Sender: [email-masked]

I see a problem here with making a content type (zipped) act like a c-t-e. 
Zipped seems fine for "attachments", that is, leaves in the MIME tree. But 
some of the requirement for making messages smaller would want to compress 
a whole message, which might be a nested multipart. At this point, zipped 
hides the lower layers. If we want a rule that say "you can't use 
app/zipped for multiparts", how does this become different than a c-t-e?

--Paul Hoffman, Director
--Internet Mail Consortium

From: "Larry Masinter" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: RE: compressed content-transfer-encoding? 
Date: Sat, 31 Jul 1999 09:44:44 PDT
Importance: Normal
Sender: [email-masked]

> > Even though application/zip is lame, other mechanisms might be
> > worse. Maybe we should look harder at providing what would be
> > necessary to make 'application/zip' actually work, e.g., some
> > top-level indication of what's actually in the package?
> 
> this has the same deployment barrier as adding a new 
> content-transfer-encoding - in either case the mime mail
> reader needs to know what to do with the extra parameter
> or the new c-t-e.

I'm not sure this is true. Unaware MIME mailers will just
see they have application/zip, that they either recognize
or not, and users of existing MIME mail programs have simple 
way of configuring their mail reader to at do something
sensible with it.

Adding a new content-transfer-encoding has a more serious
deployment problem, because most deployed systems don't have
any kind of extensibility built in for CTE.

Mailing around zip files is common practice. Many people are
used to dealing with getting a zip file. Leveraging this
just means making the common practice easier to accomplish
for senders, and less awkward for receivers; it's a product
enhancement that mail client vendors could add today.

In the menu for 'attach file', it could just have a check
box for 'send zipped', for example, and a setting for making
'send zipped' the default for attachments, or for attachments
that aren't known to already have better media-specific
compression.

From: Keith Moore <[email-masked]>
To: "Larry Masinter" <[email-masked]>
cc: "Keith Moore" <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Sat, 31 Jul 1999 12:02:36 -0400
Sender: [email-masked]

> Even though application/zip is lame, other mechanisms might be
> worse. Maybe we should look harder at providing what would be
> necessary to make 'application/zip' actually work, e.g., some
> top-level indication of what's actually in the package?

this has the same deployment barrier as adding a new 
content-transfer-encoding - in either case the mime mail
reader needs to know what to do with the extra parameter
or the new c-t-e.

From: "Larry Masinter" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>
Subject: RE: compressed content-transfer-encoding? 
Date: Sat, 31 Jul 1999 09:01:00 PDT
Importance: Normal
Sender: [email-masked]

> but the real problem with the application/zip hack is not the format,
> it's that it's a very lame way of adding compression to MIME.

Very cogent argument. Perhaps you meant to say that using the content-type
to indicate compression format hides the actual content.

These days, anti-virus scanners seem to have a feature for 'scan ZIP
files too', so, even though it's lame, the auxiliary mechanisms seem
to be making their way through the infrastructure.

And Java ships with Zip-file 'file system' support.

Even though application/zip is lame, other mechanisms might be
worse. Maybe we should look harder at providing what would be
necessary to make 'application/zip' actually work, e.g., some
top-level indication of what's actually in the package?

Larry

From: Keith Moore <[email-masked]>
To: [email-masked] (Kai Henningsen)
cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Sat, 31 Jul 1999 11:26:08 -0400
Sender: [email-masked]

> > even so, I don't think zip qualifies.  it's common, but
> > certainly not ubiquitous, and it's mostly used on a single
> > vendor's platforms.
> 
> Well, it's difficult finding a platform zip has not been ported to. 

indeed, you can find a zip program for almost any platform in existence.  
but that doesn't mean it's widely used on more than one of those platforms.

(similar things could be said for most comprssion/archive formats)

but the real problem with the application/zip hack is not the format,
it's that it's a very lame way of adding compression to MIME.

Keith

Date: 31 Jul 1999 12:40:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: compressed content-transfer-encoding?
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Keith Moore)  wrote on 30.07.99 in <[email-masked]>:

> > My definition of a standard is "a common format or protocols
> > used by many different interworking products from different
> > vendors".
>
> even so, I don't think zip qualifies.  it's common, but
> certainly not ubiquitous, and it's mostly used on a single
> vendor's platforms.

Well, it's difficult finding a platform zip has not been ported to. OTOH,  
I have no idea how widespread MIME support for application/zip is.

As a data point, it's in my /etc/mime.types (and, incidentally, /etc/mime- 
magic). That's a Debian/GNU Linux system.

And ISTR that the first commercial vendor to ship their OS with an  
unzipping lib was IBM (OS/2), and that M$ has still not done so.

MfG Kai

From: Keith Moore <[email-masked]>
To: Jacob Palme <[email-masked]>
cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Fri, 30 Jul 1999 15:47:46 -0400
Sender: [email-masked]

> My definition of a standard is "a common format or protocols
> used by many different interworking products from different
> vendors".

even so, I don't think zip qualifies.  it's common, but 
certainly not ubiquitous, and it's mostly used on a single
vendor's platforms.

Keith

From: Keith Moore <[email-masked]>
To: Dan Wing <[email-masked]>
cc: [email-masked]
Subject: Re: IMAP/POP compression for slow links 
Date: Fri, 30 Jul 1999 15:45:54 -0400
Sender: [email-masked]

>  I would argue that few people run SMTP servers on slow links

really?  I've heard lots of complaints from people about long delays 
when trying to submit messages over a dialup line, using SMTP.

If TLS can do the compression (and offhand I don't remember if it
does this), I see no reason to define a separate layer for this purpose.

Keith

Date: Fri, 30 Jul 1999 12:38:50 -0700 (PDT)
From: Dan Wing <[email-masked]>
To: [email-masked]
Subject: IMAP/POP compression for slow links
Sender: [email-masked]

As someone else pointed out, POP and IMAP is where the users feel an
impact.  I would argue that few people run SMTP servers on slow links, and
while most users at work are on fast links (typically ethernet), most
users at home or on the road are on slow links.

Thus, perhaps an IMAP (and POP) extension to allow on-the-fly compression
would give the biggest bang for the buck.  It has an additional advantage
of not causing the interoperability problems with a new Content-Encoding.

Such an extension would be useful even after RESCAP and some kind of
Content-Encoding compression were being deployed and would provide a
benefit to users even if the sender didn't support RESCAP or the new
Content-Encoding compression.

-Dan Wing

Date: Fri, 30 Jul 1999 21:25:58 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

At 10.28 -0400 99-07-29, Keith Moore wrote:
> > We already have a standard for sending compressed data in e-mail.
> > We have an IANA registered content-type application/zip.
>
>application/zip is not a standard.

It depends on the definition of a standard. If you define a
standard the way RFC 2418 defines "Internet standard" then
it is not a standard. But then neither ASCII nor UNICODE
nor HTML 4.0 are standards!

And not even HTTP is a standard, it is only a draft standard!

I would prefer that standards developed by IETF are labelled
IETF standards and not Internet standards, since the term
"Internet standard" may wrongly give the impression that
these are the only standards for the Internet.

My definition of a standard is "a common format or protocols
used by many different interworking products from different
vendors". With that definiton of "standard" certainly
application/zip is a standard.
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

From: Keith Moore <[email-masked]>
To: "Tim Kehres" <[email-masked]>
cc: [email-masked], "Jacob Palme" <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 13:47:01 -0400
Sender: [email-masked]

> >With compression in the application layer, an attachment will
> >be compressed by the original sender, and not uncompressed
> >again until by the final recipient.
> 
> 
> In addition however the receiving UA will need to have compatible
> capabilities with the sending UA.   If this is not the case, the message may
> get there, but will be rendered unusable.   I suspect that this is the
> problem with breaking software that Keith has been suggesting.

there are two kinds of brokenness that this threatens to introduce:

1. brokenness due to lack of backward compatibility
2. brokenness due to software bugs as a result of increased
   complexity.

email is already too unreliable; the last thing I want to
do is make MTAs more complex than they already are (thus further
decreasing reliability) for a dubious benefit.  

note that some approaches to compression are more reliable than 
others - using a separate per-hop negotiated compression layer 
on top of SMTP strikes me as more reliable than on-the-fly 
translation of content-transfer-encodings.  particularly if 
the separate compression layer has built in integrity checking.

Keith

From: "Tim Kehres" <[email-masked]>
To: <[email-masked]>, "Jacob Palme" <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
Date: Fri, 30 Jul 1999 01:19:27 +0800
Sender: [email-masked]

Jacob,

>With compression in the application layer, an attachment will
>be compressed by the original sender, and not uncompressed
>again until by the final recipient.

In addition however the receiving UA will need to have compatible
capabilities with the sending UA.   If this is not the case, the message may
get there, but will be rendered unusable.   I suspect that this is the
problem with breaking software that Keith has been suggesting.

The advantage of this approach is that you (may) save in local and transient
storage.

>With compression in the transport layer, the attachment
>will be compressed and uncompressed for each store-and-forward
>step.

In this situation the compression is transparent to both end UA's, and it's
application is dependent upon the negioated capabilities of each transport
link.  Advantages include not having to modify any UA's or have knowledge of
recipient UA capabilities.   Disadvantage is that there is no gain on the
local storage side.

Seems like both approaches may have their merits, depending upon what
problem you are trying to solve at a given time.

Best Regards,

-- Tim

From: Keith Moore <[email-masked]>
To: [email-masked] (Yutaka Sato =?ISO-2022-JP?B?GyRAOjRGI0stGyhK?=)
cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 12:43:54 -0400
Sender: [email-masked]

> I think the standard should satisfy followings:
>  - Content-Type must indicate the type of the body properly (of course)
>  - any combination of Content-Type and C-T-Encoding should be possible
>  - any order of recursive (nested) C-T-Encodings should be applicable

oh no, not this argument again...

> Maybe I reinvented something rejected in past but I'm curious why using
> message/rfc822 for nested encoding is not good.  

the more levels of encoding you have, the less chance you
have of being able to interoperate.

Keith

Date: Fri, 30 Jul 99 01:29:31 JST
To: [email-masked]
Subject: Re: compressed content-transfer-encoding?
From: [email-masked] (Yutaka Sato =?ISO-2022-JP?B?GyRAOjRGI0stGyhK?=)
Organization: 
Sender: [email-masked]

In message <[email-masked]> on 07/27/99(23:56:14)
you "Mark Horton" <[email-masked]> wrote:
 |It seems that a new MIME standard for content-transfer-encoding that
 |would indicate a compressed base64 type ala gzip could be nice.

I think the standard should satisfy followings:
 - Content-Type must indicate the type of the body properly (of course)
 - any combination of Content-Type and C-T-Encoding should be possible
 - any order of recursive (nested) C-T-Encodings should be applicable
Maybe I reinvented something rejected in past but I'm curious why using
message/rfc822 for nested encoding is not good.  For example, I think
we can send gziped text encoded in base64 like this:

    Content-Type: message/mime
    Content-Transfer-Encoding: base64

    <base64-encoded-body>

where the <base64-encoded-body> is decoded into a MIME message like this:

    Content-Type: text/plain
    Content-Transfer-Encoding: x-gzip

    <gzip-encoded-body>

Cheers,
Yutaka
--
Yutaka Sato <[email-masked]> http://www.etl.go.jp/~ysato/   @ @ 
Computer Science Division, Electrotechnical Laboratory      ( - )
1-1-4 Umezono, Tsukuba, Ibaraki, 305-8568 Japan            _<   >_

To: Keith Moore <[email-masked]>
Cc: Jacob Palme <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
From: [email-masked]
Date: Thu, 29 Jul 1999 11:22:56 -0400
Sender: [email-masked]

--==_Exmh_1100814738P
Content-Type: text/plain; charset=us-ascii

On Thu, 29 Jul 1999 10:28:37 EDT, Keith Moore said:
> > We already have a standard for sending compressed data in e-mail.
> > We have an IANA registered content-type application/zip.
> 
> application/zip is not a standard.

Not only is it not a standard, but it loses in the translation.

Let's say I start with a Microsoft Word application.  Currently, I
can attach that as an application/msword, and at the remote end, the MUA
will be able to intuit proper handling of the *actual file* from that.

An application/zip loses that.  It basically renders any object
as unidentifiable as an application/octet-stream.

There's also a security issue, in that I can be confident that a MIME
bodypart that is an application/msword is *one* file, the name of which
I can either predict and check, or control.

It's a lot harder to make *real* sure that you're not unzipping things
you didn't want to.  

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_1100814738P
Content-Type: application/pgp-signature

--==_Exmh_1100814738P--

To: M Horton <[email-masked]>
Cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
From: [email-masked]
Date: Thu, 29 Jul 1999 11:18:05 -0400
Sender: [email-masked]

--==_Exmh_1094026038P
Content-Type: text/plain; charset=us-ascii

On Wed, 28 Jul 1999 16:13:28 EDT, M Horton <[email-masked]>  said:
> (3) Have a phased deployment of clients.  We choose a date, say, 2
> years off, by when we expect most clients will be upgraded.  Each
> client has a switch setting for whether to compress binary attachments,
> with 3 settings:
> 	Always compress
> 	Never compress
> 	Compress if the send date is after <the chosen date>
> The default setting could be the third one.

Gaak. *NO*.

I found a Sendmail 5.65 on our campus this month.

Think about that. ;)

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_1094026038P
Content-Type: application/pgp-signature

--==_Exmh_1094026038P--

To: "Tim Kehres" <[email-masked]>
Cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
From: [email-masked]
Date: Thu, 29 Jul 1999 11:16:38 -0400
Sender: [email-masked]

--==_Exmh_1091491590P
Content-Type: text/plain; charset=us-ascii

On Thu, 29 Jul 1999 04:24:24 +0800, "Tim Kehres" said:
> Above you also mention the online storage of the messages.   What is the
> intent here?   Are we trying to save in disk space or bandwidth utilization
> when the attachment is in transit.   I was assuming the later.    Saving a
> couple hundred meg of disk space with the price of drives these days hardly
> seems worth the effort.

I mentioned online storage only because I had statistics on the compression.
What an MUA does in its message store is of course its own business. ;)

I'm also distressed at this cavalier "with the cost of..." trend of late.

Some places run on tight budgets or have other constraints.  I turned
on disk compression on my worstation, and got back about 30% of the /home
space.  This saved me from having to buy a new disk drive.  So right
there, we've saved $200 just on the cost of the drive.  But there's more.

Remember to factor in the cost of my downtime while I back up my machine,
take it down, do all the needed recabling, replace the drive, boot it
up, and restore all the data from tape.  Add the cost of most of a day's
work for me.  Add in the cost of *tomorrow* being shot because I didn't
get stuff done today, so tomorrow I'm digging out from under today's backlog.

Suddenly, worrying about efficient use of disk space starts looking a
lot better - and makes a CPU and/or memory upgrades a lot cheaper in
comparison.

And I've *got* disk compression software.  I don't have CPU compression
software. ;)

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_1091491590P
Content-Type: application/pgp-signature

--==_Exmh_1091491590P--

From: Keith Moore <[email-masked]>
To: Jacob Palme <[email-masked]>
cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 10:28:37 -0400
Sender: [email-masked]

> We already have a standard for sending compressed data in e-mail.
> We have an IANA registered content-type application/zip.

application/zip is not a standard.

Date: Thu, 29 Jul 1999 14:07:41 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

We already have a standard for sending compressed data in e-mail.
We have an IANA registered content-type application/zip.

What more is needed? That mailers start using it?
Is there a need for an attribute to "Content-Type:application/zip"
with the name "Uncompressed-Content-Type"?.
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

Date: Thu, 29 Jul 1999 11:22:45 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: RE: compressed content-transfer-encoding?
Sender: [email-masked]

At 11.57 -0700 99-07-27, Larry Masinter wrote:
>HTTP (RFC 2616) added a different transformational layer
>(Content-Encoding) to avoid combinatorial explosion with different
>transfer-encodings (transfer-encodings were also kept distinct
>from content-transfer-encodings). Valid content-coding tokens
>include "gzip", "compress" and "deflate".

After thinking more about this, I have come to the conclusion
that compression should be automatic and supported by the
standards. The reason for this is that this will make things
easier for the users. The users need not ever see that information
is compressed during transmission.

The issue is whether compression should be done in the application
layer, with special compression headers, etc., or in lower layers.

For example, and e-mail message is usually forwarded through
at least two store-and-forward MTAs:

Original  ->  Local MTA   ->  Local MTA      -> Final
sender        for sender      for recipient     recipient

With compression in the application layer, an attachment will
be compressed by the original sender, and not uncompressed
again until by the final recipient.

With compression in the transport layer, the attachment
will be compressed and uncompressed for each store-and-forward
step.
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

From: Keith Moore <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Keith Moore <[email-masked]>, Tim Kehres <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Wed, 28 Jul 1999 20:46:43 -0400
Sender: [email-masked]

> > my point is that even the "type-independent compressions" (LZW,
> > deflate, etc)  are at best ineffective (and at worse pessimal)
> > when applied to different kinds of media types for which they
> > were designed.  you therefore don't want to gratuitously apply
> > them to random content-types.
> 
> Actually this isn't true -- for one thing, there's still an advantage to 
> be had by compressing base64 at the transport level. 

right, but I was talking about MIME compression. if we compress at
the transport level, we might as well just use ssl.

> And for another, the
> type-independent conversions sometimes do manage to squeeze a little 
> more juice out out that the type-specific conversions didn't manage. 

yes, though it's often prety marginal - my guess is that it's not worth
the cost of incompatibility.  (again, talking about MIME compression)

> And when that's not
> possible, they are smart enough to tell this is the case and opt out; the
> result is a little CPU lost on the sending side (very cheap), effectively no
> bloat, and no significant CPU loss on the receiver.

basically true, for newer algorithms.

Keith

Date: 28 Jul 1999 17:50:32 -0400
From: Timothy L Martin <[email-masked]>
To: Tim Kehres <[email-masked]>
CC: [email-masked]
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

> > capability.  It would be tough on the MTA's, but with CPU and disk speeds
> > increasing, it might be feasible.   Anyway, just a thought...
> 
> I've heard proposals to possibly do this through a SASL layer.

The main issue with this is that current protocols that support SASL
only allow for one SASL layer at a time so you could get either
compression or kerberos, not both. I still think this could be useful
in certain circumstances and would be willing to implement it if
anyone were to write a SASL compression draft.

Date: Wed, 28 Jul 1999 14:23:56 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
To: Keith Moore <[email-masked]>
Cc: Ned Freed <[email-masked]>, Keith Moore <[email-masked]>, Tim Kehres <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Sender: [email-masked]

> my point is that even the "type-independent compressions" (LZW,
> deflate, etc)  are at best ineffective (and at worse pessimal)
> when applied to different kinds of media types for which they
> were designed.  you therefore don't want to gratuitously apply
> them to random content-types.

Actually this isn't true -- for one thing, there's still an advantage to be had
by compressing base64 at the transport level. And for another, the
type-independent conversions sometimes do manage to squeeze a little more juice
out out that the type-specific conversions didn't manage. And when that's not
possible, they are smart enough to tell this is the case and opt out; the
result is a little CPU lost on the sending side (very cheap), effectively no
bloat, and no significant CPU loss on the receiver.

The quality of these things really has gone up over the years, so with the
proper use of modern options pessimal outcomes are eliminated.

				Ned

Date: Wed, 28 Jul 1999 16:24:48 -0400
From: Tony Hansen <[email-masked]>
Organization: AT&T Laboratories
To: Tim Kehres <[email-masked]>
CC: Keith Moore <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

Tim Kehres wrote:
> 
> >> It seems that a new MIME standard for content-transfer-encoding that
> >> would indicate a compressed base64 type ala gzip could be nice.
> >> Creative minds might even improve the efficiency of base64 at the
> >> same time, if we don't have to worry about translation into EBCDIC
> >> anymore.
> >
> >it's been discussed many times; afaik the biggest problem is that nobody
> >has bothered to write up a concrete proposal.  the second biggest problem,
> >of course, is that it would break lots of existing software.
> 
> Just curious - has the idea of doing on the fly compression at the ESMTP
> level ever been considered?   This would have the advantage of not breaking
> any of the upper layers, and would only be enabled between MTA's with the
> capability.  It would be tough on the MTA's, but with CPU and disk speeds
> increasing, it might be feasible.   Anyway, just a thought...

I've heard proposals to possibly do this through a SASL layer.

	Tony Hansen
	[email-masked]

From: "Tim Kehres" <[email-masked]>
To: "Keith Moore" <[email-masked]>, <[email-masked]>
Cc: "Mark Horton" <[email-masked]>, <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 04:24:24 +0800
Sender: [email-masked]

>On Wed, 28 Jul 1999 15:03:23 EDT, Keith Moore said:
>> it's been discussed many times; afaik the biggest problem is that nobody
>> has bothered to write up a concrete proposal.  the second biggest
problem,
>> of course, is that it would break lots of existing software.
>
>That, and the fact that currently, the average text/plain being sent
>around is relatively small (2-3K or so) and won't be a BIG win (you can't
>save mor than 3 K, and that's only 2 packets on an ethernet ;)
>
>The things that chew up the bandwidth are things like .GIF, .JPG,
>etc attachements, which usually tend to have some compression already
>done on them.  Now, if you have some big spreadsheets from some big
>company that specializes in bloatware, perhaps the right thing to do
>is convince them to make it take less disk space.  Yes, disk is cheap,
>but that hardly justifies intentional waste....
>
>Has anybody done any studies at all on whether said compression would
>actually *win* us enough to be worth it?  As a data point, my MH folders
>live on a compressed file system (each 4K block is LZ-compressed
individually),
>and takes about 110M compressed and 198M uncompressed.  *HOWEVER*, a *very*
>large chunk of that is Received: headers and the like, which would NOT
>be compressible...

As you say, it is highly dependent upon the attachment type.   File types
that are already in a compressed state like JPEG's as you reference above
won't buy you anything.  On the other hand, at least in our environment, we
tend to send around a lot of documentation, in the form of Microsoft Word,
Adobe FrameMaker, or PDF formats, which compress rather nicely.   Images in
TIFF format are also highly compressible.  I would suspect that as the usage
of HTML in email increases, you might get some benefit here as well, but of
course only for the larger sized messages.

Due to the dependency on the attachment types, I suspect that what might be
a significant win for one environment would make no difference in another.
In reference to your comment above about the bloatware and convincing people
to use less space, I'm not sure what you have in mind here.  I'm not aware
of any simple way to have for instance your Excel, Word, or Frame files
automagically compressed.

Above you also mention the online storage of the messages.   What is the
intent here?   Are we trying to save in disk space or bandwidth utilization
when the attachment is in transit.   I was assuming the later.    Saving a
couple hundred meg of disk space with the price of drives these days hardly
seems worth the effort.

Best Regards,

Tim Kehres
International Messaging Associates
http://www.ima.com

From: Keith Moore <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Keith Moore <[email-masked]>, Tim Kehres <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Wed, 28 Jul 1999 16:21:04 -0400
Sender: [email-masked]

my point is that even the "type-independent compressions" (LZW, 
deflate, etc)  are at best ineffective (and at worse pessimal) 
when applied to different kinds of media types for which they 
were designed.  you therefore don't want to gratuitously apply 
them to random content-types.

Keith

Date: Wed, 28 Jul 1999 16:13:28 -0400
From: M Horton <[email-masked]>
Organization: Lucent Technologies
To: [email-masked]
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

> Just curious - has the idea of doing on the fly compression at the ESMTP
> level ever been considered?

I think there are two major problems this proposal tries to solve:

(1) Big attachments take forever to download over a slow link with
POP or IMAP, or to submit with SMTP.

(2) Big attachments take up a lot of disk space on the mail server
or in mail folders where you save them.

Enhancing SMTP doesn't solve either problem.

> We need a deployment mechanism first. And that's precisely what we're
> trying to get through RESCAP.

Not being familiar with RESCAP, do you have a URL for it?

Some ideas for deployment:

(1) Provide a free tool that will convert the compressed format to
the uncompressed format, open source and also compiled for the
various platforms.

(2) Put up some mail relays on the net.  You forward your message to
a mail relay - it returns it to you with uncompressed attachments.

(3) Have a phased deployment of clients.  We choose a date, say, 2
years off, by when we expect most clients will be upgraded.  Each
client has a switch setting for whether to compress binary attachments,
with 3 settings:
	Always compress
	Never compress
	Compress if the send date is after <the chosen date>
The default setting could be the third one.

In addition, the Compose window should let you override the default
setting for a specific message.

> That, and the fact that currently, the average text/plain being sent
> around is relatively small (2-3K or so) and won't be a BIG win (you can't
> save mor than 3 K, and that's only 2 packets on an ethernet ;)
> 
> The things that chew up the bandwidth are things like .GIF, .JPG,
> etc attachements, which usually tend to have some compression already
> done on them.  Now, if you have some big spreadsheets from some big
> company that specializes in bloatware, perhaps the right thing to do
> is convince them to make it take less disk space.  Yes, disk is cheap,
> but that hardly justifies intentional waste....

I would think one would want a setting in the client saying "only
compress attachments if they are larger than ___ KB."  My experience
is that MS Office files are the usual large attachments that cause
problems.  While I am hopeful that Microsoft will eventually have a
reasonably small default Office format, their web page says Office 2000
files are the same format as Office 97, and that their HTML versions
are actually larger than the proprietary format.

> Has anybody done any studies at all on whether said compression would
> actually *win* us enough to be worth it?  As a data point, my MH folders
> live on a compressed file system (each 4K block is LZ-compressed
> individually),
> and takes about 110M compressed and 198M uncompressed.  *HOWEVER*, a *very*
> large chunk of that is Received: headers and the like, which would NOT
> be compressible...

I tried compressing (with winzip, which seems to be universally available
on PCs) a lot of large Office files I have around.  The results were
spectacular - 90% compression was common.  I really had to hunt to find
files that compressed to the expected 50%.

I would guess that GZip would do even better.

	Mark

From: "Tim Kehres" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: "Mark Horton" <[email-masked]>, <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 04:09:31 +0800
Sender: [email-masked]

>> >aarrgh.  SMTP is already complex enough.  the last thing I want to see
>> >is more complex MTAs adding more failure cases.
>>
>>
>> I was thinking in terms of an ESMTP extension.   In any event, by your
own
>> admission, doing it as a MIME type would break a lot of software at the
end
>> points.  At least this way it is only attempted between mutually
consenting
>> MTA's and is transparent to the end users.
>
>an MTA that transparently corrupts your mail isn't necessarily a good
thing.

Yes, of course.   I was not advocating the populating of the world with
broken software.  :-)   :-)

I've not had a chance to review the RFC that Ned made reference to (TLS STMP
Extension), but on the surface it sounds like the capabilities that I was
suggesting anyway.

Thanks and Best Regards,

Tim Kehres
International Messaging Associates
http://www.ima.com

Date: Wed, 28 Jul 1999 13:05:40 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
To: Keith Moore <[email-masked]>
Cc: Tim Kehres <[email-masked]>, Keith Moore <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Sender: [email-masked]

> > >aarrgh.  SMTP is already complex enough.  the last thing I want to see
> > >is more complex MTAs adding more failure cases.
> >
> >
> > I was thinking in terms of an ESMTP extension.   In any event, by your own
> > admission, doing it as a MIME type would break a lot of software at the end
> > points.  At least this way it is only attempted between mutually consenting
> > MTA's and is transparent to the end users.

> an MTA that transparently corrupts your mail isn't necessarily a good thing.

> note also that compression algorithms tend to be specific to particular
> kinds of content - what works well with text or object code typically
> doesn't work well with images, audio, or video.

I prefer to characterize it this way: There are type-specific compressions,
which tend to be built into the media types they are appropriate for and which
also may not yield the same output as input, and type-independent compressions,
which tend to be applied on top of rather than inside of media types and which
always yield the same output as input.

We're only talking about the latter sort of compression here. Use of the former
is a solved problem, and hence a red herring in this context.

				Ned

From: Keith Moore <[email-masked]>
To: "Tim Kehres" <[email-masked]>
cc: "Keith Moore" <[email-masked]>, "Mark Horton" <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Wed, 28 Jul 1999 15:58:10 -0400
Sender: [email-masked]

> >aarrgh.  SMTP is already complex enough.  the last thing I want to see
> >is more complex MTAs adding more failure cases.
> 
> 
> I was thinking in terms of an ESMTP extension.   In any event, by your own
> admission, doing it as a MIME type would break a lot of software at the end
> points.  At least this way it is only attempted between mutually consenting
> MTA's and is transparent to the end users.

an MTA that transparently corrupts your mail isn't necessarily a good thing.

note also that compression algorithms tend to be specific to particular
kinds of content - what works well with text or object code typically 
doesn't work well with images, audio, or video.

Keith

To: Keith Moore <[email-masked]>
Cc: "Mark Horton" <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
From: [email-masked]
Date: Wed, 28 Jul 1999 15:54:57 -0400
Sender: [email-masked]

--==_Exmh_-448553088P
Content-Type: text/plain; charset=us-ascii

On Wed, 28 Jul 1999 15:03:23 EDT, Keith Moore said:
> it's been discussed many times; afaik the biggest problem is that nobody
> has bothered to write up a concrete proposal.  the second biggest problem,
> of course, is that it would break lots of existing software.

That, and the fact that currently, the average text/plain being sent
around is relatively small (2-3K or so) and won't be a BIG win (you can't
save mor than 3 K, and that's only 2 packets on an ethernet ;)

The things that chew up the bandwidth are things like .GIF, .JPG,
etc attachements, which usually tend to have some compression already
done on them.  Now, if you have some big spreadsheets from some big
company that specializes in bloatware, perhaps the right thing to do
is convince them to make it take less disk space.  Yes, disk is cheap,
but that hardly justifies intentional waste....

Has anybody done any studies at all on whether said compression would
actually *win* us enough to be worth it?  As a data point, my MH folders
live on a compressed file system (each 4K block is LZ-compressed individually),
and takes about 110M compressed and 198M uncompressed.  *HOWEVER*, a *very*
large chunk of that is Received: headers and the like, which would NOT
be compressible...

If I get ambitious tonight, I'll see what the compression of the bodyparts
ends up being..

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_-448553088P
Content-Type: application/pgp-signature

--==_Exmh_-448553088P--

Date: Wed, 28 Jul 1999 12:52:33 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
To: Tim Kehres <[email-masked]>
Cc: Keith Moore <[email-masked]>, Mark Horton <[email-masked]>, [email-masked]
Sender: [email-masked]

> > > It seems that a new MIME standard for content-transfer-encoding that
> > > would indicate a compressed base64 type ala gzip could be nice.
> > > Creative minds might even improve the efficiency of base64 at the
> > > same time, if we don't have to worry about translation into EBCDIC
> > > anymore.

> > it's been discussed many times; afaik the biggest problem is that nobody
> > has bothered to write up a concrete proposal.  the second biggest problem,
> > of course, is that it would break lots of existing software.

> Just curious - has the idea of doing on the fly compression at the ESMTP
> level ever been considered?

Yes, but given the existance of the TLS SMTP extension, which can 
negotiate compression, why bother defining another mechanism?

> This would have the advantage of not breaking
> any of the upper layers, and would only be enabled between MTA's with the
> capability.  It would be tough on the MTA's, but with CPU and disk speeds
> increasing, it might be feasible.   Anyway, just a thought...

Compression is a nonissue compared to crypto. But in general we need crypto a
lot more than we need to save bandwidth...

				Ned

Date: Wed, 28 Jul 1999 12:49:10 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
To: Keith Moore <[email-masked]>
Cc: Mark Horton <[email-masked]>, [email-masked]
Sender: [email-masked]

> > It seems that a new MIME standard for content-transfer-encoding that
> > would indicate a compressed base64 type ala gzip could be nice.
> > Creative minds might even improve the efficiency of base64 at the
> > same time, if we don't have to worry about translation into EBCDIC
> > anymore.

> it's been discussed many times; afaik the biggest problem is that nobody
> has bothered to write up a concrete proposal.  the second biggest problem,
> of course, is that it would break lots of existing software.

I actually have it written up, but have never bothered to release the
specification because of the deployment problem.

We need a deployment mechanism first. And that's precisely what we're trying
to get through RESCAP.

				Ned

From: "Tim Kehres" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: "Mark Horton" <[email-masked]>, <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 03:49:10 +0800
Sender: [email-masked]

>> Just curious - has the idea of doing on the fly compression at the ESMTP
>> level ever been considered?
>
>aarrgh.  SMTP is already complex enough.  the last thing I want to see
>is more complex MTAs adding more failure cases.

I was thinking in terms of an ESMTP extension.   In any event, by your own
admission, doing it as a MIME type would break a lot of software at the end
points.  At least this way it is only attempted between mutually consenting
MTA's and is transparent to the end users.

Best Regards,

Tim Kehres
International Messaging Associates
http://www.ima.com

From: Keith Moore <[email-masked]>
To: "Tim Kehres" <[email-masked]>
cc: "Keith Moore" <[email-masked]>, "Mark Horton" <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Wed, 28 Jul 1999 15:35:23 -0400
Sender: [email-masked]

> Just curious - has the idea of doing on the fly compression at the ESMTP
> level ever been considered?

aarrgh.  SMTP is already complex enough.  the last thing I want to see
is more complex MTAs adding more failure cases.

Date: Thu, 29 Jul 1999 00:35:53 +0500
From: "Bonatti, Chris" <[email-masked]>
Organization: IECA, Inc.
To: Keith Moore <[email-masked]>
CC: Mark Horton <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

Keith,

Actually, I suspect that you have it backwards because there is a dependency
involved.  I think that a lot of folks would be willing to write such a
proposal (myself included) except for the second problem.  ;-)  We are victims
of our own success.  We can't afford to think of better approaches because we
can't afford to have a big departure for Web:TNG or SMTP:TNG.

Chris

__________________

Keith Moore wrote:

> > It seems that a new MIME standard for content-transfer-encoding that
> > would indicate a compressed base64 type ala gzip could be nice.
> > Creative minds might even improve the efficiency of base64 at the
> > same time, if we don't have to worry about translation into EBCDIC
> > anymore.
>
> it's been discussed many times; afaik the biggest problem is that nobody
> has bothered to write up a concrete proposal.  the second biggest problem,
> of course, is that it would break lots of existing software.
>
> Keith

From: "Tim Kehres" <[email-masked]>
To: "Keith Moore" <[email-masked]>, "Mark Horton" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: compressed content-transfer-encoding? 
Date: Thu, 29 Jul 1999 03:26:55 +0800
Sender: [email-masked]

>> It seems that a new MIME standard for content-transfer-encoding that
>> would indicate a compressed base64 type ala gzip could be nice.
>> Creative minds might even improve the efficiency of base64 at the
>> same time, if we don't have to worry about translation into EBCDIC
>> anymore.
>
>it's been discussed many times; afaik the biggest problem is that nobody
>has bothered to write up a concrete proposal.  the second biggest problem,
>of course, is that it would break lots of existing software.

Just curious - has the idea of doing on the fly compression at the ESMTP
level ever been considered?   This would have the advantage of not breaking
any of the upper layers, and would only be enabled between MTA's with the
capability.  It would be tough on the MTA's, but with CPU and disk speeds
increasing, it might be feasible.   Anyway, just a thought...

Best Regards,

Tim Kehres
International Messaging Associates
http://www.ima.com

From: Keith Moore <[email-masked]>
To: "Mark Horton" <[email-masked]>
cc: [email-masked]
Subject: Re: compressed content-transfer-encoding? 
Date: Wed, 28 Jul 1999 15:03:23 -0400
Sender: [email-masked]

> It seems that a new MIME standard for content-transfer-encoding that
> would indicate a compressed base64 type ala gzip could be nice.
> Creative minds might even improve the efficiency of base64 at the
> same time, if we don't have to worry about translation into EBCDIC
> anymore.

it's been discussed many times; afaik the biggest problem is that nobody
has bothered to write up a concrete proposal.  the second biggest problem,
of course, is that it would break lots of existing software.

Keith

Date: Wed, 28 Jul 1999 05:34:53 +0500
From: "Bonatti, Chris" <[email-masked]>
Organization: IECA, Inc.
To: Larry Masinter <[email-masked]>
CC: Mark Horton <[email-masked]>, [email-masked]
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

Congratulations.  We've reinvented the OSI Presentation layer.   :-|

Kidding aside.  This requirement (?) has also been discussed in the context
of X.400, but it has never motivated anybody enough to make it happen.  My
feeling is that RFC 2616 took basically the right approach, though.

What we are sorely in need of are some standardized IETF middleware
applications that do things like compression, digital signature, etc. for
any client application.  There are a number of good models for this that
could be drawn on.  However, I don't think that this will happen until a
critical mass of people get fed up with solving every problem n times (for n
applications).

Chris

 --------------------------------
 | International Electronic     |
 | Communication Analysts, Inc. |
 | Christopher D. Bonatti       |
 | Principal Engineer           |
 | [email-masked]            |
 | Tel: 301-208-2349            |
 --------------------------------

___________________

Larry Masinter wrote:

> > Is there an RFC (or movement toward one) for a compressed encoding
> > within MIME?
> >
> > There seems to be a lot of interest lately in compressing attachments.
> > End users are encouraged to zip Office files before mailing them,
> > and at least one product (MaxCompression) does this automatically
> > (in a proprietary way.)  Even with modem compression, there seems to
> > be some gain from this, and the disk storage implications on the mail
> > server are also interesting.
> >
> > It seems that a new MIME standard for content-transfer-encoding that
> > would indicate a compressed base64 type ala gzip could be nice.
> > Creative minds might even improve the efficiency of base64 at the
> > same time, if we don't have to worry about translation into EBCDIC
> > anymore.
>
> HTTP (RFC 2616) added a different transformational layer
> (Content-Encoding) to avoid combinatorial explosion with different
> transfer-encodings (transfer-encodings were also kept distinct
> from content-transfer-encodings). Valid content-coding tokens
> include "gzip", "compress" and "deflate".
>
> It might be possible to extend Content-Encoding to work with mail.
>
> And if you want to avoid Base64 in mail, well, there's BINARYMIME
> (RFC 1830).
>
> Larry
> --
> http://www.parc.xerox.com/masinter

From: "Larry Masinter" <[email-masked]>
To: "Mark Horton" <[email-masked]>
Cc: <[email-masked]>
Subject: RE: compressed content-transfer-encoding?
Date: Tue, 27 Jul 1999 11:57:34 PDT
Importance: Normal
Sender: [email-masked]

> Is there an RFC (or movement toward one) for a compressed encoding
> within MIME?
> 
> There seems to be a lot of interest lately in compressing attachments.
> End users are encouraged to zip Office files before mailing them,
> and at least one product (MaxCompression) does this automatically
> (in a proprietary way.)  Even with modem compression, there seems to
> be some gain from this, and the disk storage implications on the mail
> server are also interesting.
> 
> It seems that a new MIME standard for content-transfer-encoding that
> would indicate a compressed base64 type ala gzip could be nice.
> Creative minds might even improve the efficiency of base64 at the
> same time, if we don't have to worry about translation into EBCDIC
> anymore.

HTTP (RFC 2616) added a different transformational layer
(Content-Encoding) to avoid combinatorial explosion with different
transfer-encodings (transfer-encodings were also kept distinct
from content-transfer-encodings). Valid content-coding tokens
include "gzip", "compress" and "deflate".

It might be possible to extend Content-Encoding to work with mail.

And if you want to avoid Base64 in mail, well, there's BINARYMIME
(RFC 1830).

Larry
-- 
http://www.parc.xerox.com/masinter

Date: Tue, 27 Jul 1999 20:42:35 +0200
To: [email-masked]
From: Jacob Palme <[email-masked]>
Subject: Re: compressed content-transfer-encoding?
Sender: [email-masked]

At 10.56 -0400 99-07-27, Mark Horton wrote:
>Is there an RFC (or movement toward one) for a compressed encoding
>within MIME?

HTTP has compression headers. We should find out if and how much they
are used, and why. HTTP needs compression even more than e-mail,
because people are waiting for documents to show up in real time. If
compression has not been successful in HTTP, will it be in e-mail
where the need is less than in HTTP?

--- --- Excerpts from the HTTP/1.1 specification, RFC 2616 --- ---

        content-coding   = token

    All content-coding values are case-insensitive. HTTP/1.1 uses
    content-coding values in the Accept-Encoding (section 14.3) and
    Content-Encoding (section 14.11) header fields. Although the value
    describes the content-coding, what is more important is that it
    indicates what decoding mechanism will be required to remove the
    encoding.

    The Internet Assigned Numbers Authority (IANA) acts as a registry for
    content-coding value tokens. Initially, the registry contains the
    following tokens:

    gzip An encoding format produced by the file compression program
         "gzip" (GNU zip) as described in RFC 1952 [25]. This format is a
         Lempel-Ziv coding (LZ77) with a 32 bit CRC.

    compress
         The encoding format produced by the common UNIX file compression
         program "compress". This format is an adaptive Lempel-Ziv-Welch
         coding (LZW).

         Use of program names for the identification of encoding formats
         is not desirable and is discouraged for future encodings. Their
         use here is representative of historical practice, not good
         design. For compatibility with previous implementations of HTTP,
         applications SHOULD consider "x-gzip" and "x-compress" to be
         equivalent to "gzip" and "compress" respectively.

    deflate
         The "zlib" format defined in RFC 1950 [31] in combination with
         the "deflate" compression mechanism described in RFC 1951 [29].

    identity
         The default (identity) encoding; the use of no transformation
         whatsoever. This content-coding is used only in the Accept-
         Encoding header, and SHOULD NOT be used in the Content-Encoding
         header.

    New content-coding value tokens SHOULD be registered; to allow
    interoperability between clients and servers, specifications of the
    content coding algorithms needed to implement a new value SHOULD be
    publicly available and adequate for independent implementation, and
    conform to the purpose of content coding defined in this section.

------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

From: "Mark Horton" <[email-masked]>
Date: Tue, 27 Jul 1999 10:56:14 -0400
To: [email-masked]
Subject: compressed content-transfer-encoding?
Sender: [email-masked]

Is there an RFC (or movement toward one) for a compressed encoding
within MIME?

There seems to be a lot of interest lately in compressing attachments.
End users are encouraged to zip Office files before mailing them,
and at least one product (MaxCompression) does this automatically
(in a proprietary way.)  Even with modem compression, there seems to
be some gain from this, and the disk storage implications on the mail
server are also interesting.

It seems that a new MIME standard for content-transfer-encoding that
would indicate a compressed base64 type ala gzip could be nice.
Creative minds might even improve the efficiency of base64 at the
same time, if we don't have to worry about translation into EBCDIC
anymore.

        Mark