Subject: Mime type statistics
From: Jacob Palme <[email-masked]>
To: IETF mailing list on e-mail headers <[email-masked]>
Date: Fri, 25 Sep 1998 13:56:22 +0200

Has anyone made any statistics on how common different mime
types are in e-mail usage?
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

Date: Thu, 24 Sep 1998 08:42:11 +0100 (BST)
From: ruth moulton <[email-masked]>
To: Harald Tveit Alvestrand <[email-masked]>
Cc: [email-masked]
Subject: Re: MIME messages without MIME-Version header
Cc: [email-masked]
Sender: [email-masked]

Harald,
thanks for the reference, but I get

  "File Not Found
  The requested URL /ietf/uuencode-no.htm was not found on this
  server."

when I tried to access it.
ruth

 > I've still got http://www.alvestrand.no/ietf/uuencode-no.html, which
 > gives Ned's description of some of the cute variants of UUENCODE
 > he's run into.
 > 
 > It's a fun read :-)
 > 
 >                Harald
 > 
 > -- 
 > Harald Tveit Alvestrand, Maxware, Norway
 > [email-masked]
 > 

-- 
================================================
Ruth Moulton            [email-masked]
Consultant              

65 Tetherdown, 
London N.10 1NH, UK     Tel:+44 181 883 5823

-- 

To: [email-masked]
Path: not-for-mail
From: John Gardiner Myers <[email-masked]>
Newsgroups: mcom.list.ietf-822
Subject: Re: MIME messages without MIME-Version header
Date: Wed, 23 Sep 1998 16:17:46 -0700
Organization: Netscape Communications Corporation
Lines: 8
NNTP-Posting-Host: 208.12.60.63
Sender: [email-masked]

Ned Freed wrote:
> But in practice the MIME aspect of this is simple enough -- put whatever you're
> calling uuencode today in a MIME part, label it with CTE x-uuencode, done.

In practice, it's even simpler.  Don't generate uuencode, generate
base64 instead.

Date: Thu, 24 Sep 1998 08:05:26 +0900
From: Antony Bowesman <[email-masked]>
To: Ned Freed <[email-masked]>
CC: ruth moulton <[email-masked]>, John Mani <[email-masked]>, [email-masked], [email-masked]
Subject: Re: MIME messages without MIME-Version header
Sender: [email-masked]

Ned Freed wrote:
> 
> The tricky part is what you're calling uuencode today. I've seen so many
> variants on the uuencode theme it isn't funny. As a practical matter I'd
> suggest you:
<snip>

I've also seen a uuencoder (don't know whose) put blank lines (2
sequential LF) in the middle of the file which caused some decoders
problems.

Rgds
-- 
Antony Bowesman
[email-masked]

Date: Wed, 23 Sep 1998 23:29:44 +0200
To: Ned Freed <[email-masked]>, ruth moulton <[email-masked]>
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
Cc: John Mani <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

I've still got http://www.alvestrand.no/ietf/uuencode-no.html, which
gives Ned's description of some of the cute variants of UUENCODE
he's run into.

It's a fun read :-)

               Harald

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Wed, 23 Sep 1998 10:44:54 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
To: ruth moulton <[email-masked]>
Cc: John Mani <[email-masked]>, [email-masked], [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> you mentioned x-uuencode - does any one know if/where there is
> a definition of this, i.e. the use of uuencode with mime ?

There is not and can never be a formal definition of this -- an x- CTE is by
definition a private, nonstandard thing.

But in practice the MIME aspect of this is simple enough -- put whatever you're
calling uuencode today in a MIME part, label it with CTE x-uuencode, done.

The tricky part is what you're calling uuencode today. I've seen so many
variants on the uuencode theme it isn't funny. As a practical matter I'd
suggest you:

(0) Don't put anything in the part other than the uuencode material. Cute
    comments confuse some decoders.
(1) Include a begin line with a fixed three digit protection of "600" and a
    filename consisting entirely of US-ASCII letters, numbers, and at
    most one period. The word "begin" needs to be in lower case.
(2) Have all lines be of length "M" except the last. Absolutely do not use
    lines longer than 76 characters.
(3) Use ` instead of space for the zero value in your encoding.
(4) Follow the last, possibly short, line with a single line of length "`"
    (otherwise known as zero) and follow that with a line containing an "end".
(5) Don't use any per-line checksums, checksums at the end.

Most uudecoders can handle this variant -- the only problematic aspect of
it may be the use of "`" rather than space, but in my opinion you are more
likely to run into trouble with messed up spaces than you are with decoders
that cannot handle "`".

What a decoder should do is much harder question, and one I simply don't have
time to get into in any detail.

And before anyone asks, the answer is yes, I've seen each and every one of
these guidelines violated more than once, and I've seen trouble arise from each
one of them.

				Ned

Date: Wed, 23 Sep 1998 09:04:41 +0100 (BST)
From: ruth moulton <[email-masked]>
To: John Mani <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MIME messages without MIME-Version header
Cc: [email-masked]
Sender: [email-masked]

John,
you mentioned x-uuencode - does any one know if/where there is
a definition of this, i.e. the use of uuencode with mime ?

ta
ruth

 > 
 > > 
 > > > Unfortunately HotMail sends out such messages - they look very much 
 > > >like MIME
 > > > messages with boundaries and mime headers and all, but they forgot to stick
 > > > in the MIME-Version header; they also use uuencode to encode binary data. 
 > > > (If the right folks are listening .. please fix this and spare us
 > > > client-writers some pain)
 > > 
 > > If they use UUENCODE, what do they put in the content-transfer-encoding
 > > field?
 > > 
 > 
 > x-uuencode.
 > 
 > 

-- 
================================================
Ruth Moulton            [email-masked]
Consultant              

65 Tetherdown, 
London N.10 1NH, UK     Tel:+44 181 883 5823

-- 

Date: Tue, 22 Sep 1998 13:57:07 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Cc: IMAP Discusson List <[email-masked]>
Sender: [email-masked]

On Tue, 22 Sep 1998, Mark Crispin wrote:
> Historical retrospective: MIME recycled the Content-Type header from an
> earlier multimedia mail specification instead of using a unique mechanism.
> To distinguish between the new and the old mechanisms, the MIME-Version
> header was invented.
> 
> Consequently, a message with a Content-Type header but not a MIME-Version
> header is not MIME.

That's entirely correct and fully compliant.

However, in practice I'd recommend treating any Content-Type header with a
"type/subtype" syntax as a MIME message.  The old version of the
Content-Type header didn't have subtypes so it is easily distinguished
from MIME Content-Type headers.

		- Chris

Date: Tue, 22 Sep 1998 13:18:03 -0700 (PDT)
From: Mark Crispin <[email-masked]>
To: John Mani <[email-masked]>
cc: [email-masked], [email-masked]
Subject: Re: MIME messages without MIME-Version header
Organization: Networks & Distributed Computing
Sender: [email-masked]

On Tue, 22 Sep 1998, John Mani wrote:
> How to handle MIME (-like) messages without the MIME-version header field ?
> 
> What do IMAP servers typically do in this case ? I believe the UW server
> considers it non-mime and returns it as a singlepart message (not sure).

Historical retrospective: MIME recycled the Content-Type header from an
earlier multimedia mail specification instead of using a unique mechanism.
To distinguish between the new and the old mechanisms, the MIME-Version
header was invented.

Consequently, a message with a Content-Type header but not a MIME-Version
header is not MIME.

-- Mark --

* RCW 19.149 notice: This email address is located in Washington State.	*
* Unsolicited commercial email may be billed $500 per message.		*
Science does not emerge from voting, party politics, or public debate.

Date: Tue, 22 Sep 1998 12:44:48 -0700 (PDT)
From: John Mani <[email-masked]>
Reply-To: John Mani <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
To: [email-masked], [email-masked]
Content-MD5: IZDNSHF/5JsPr04k17Ob+g==
Sender: [email-masked]

> > > If they use UUENCODE, what do they put in the content-transfer-encoding
> > > field?
> 
> > x-uuencode.
> 
> Thank heaven for small favors -- at least they used a common string,
> not to mention generating a CTE field to begin with!

Yup.

Given the number of hotmail users out there, this is gonna be a royal pain.
I've contacted Hotmail support about this ... Sigh :(

-john

Date: Tue, 22 Sep 1998 12:28:48 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
To: John Mani <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > If they use UUENCODE, what do they put in the content-transfer-encoding
> > field?

> x-uuencode.

Thank heaven for small favors -- at least they used a common string,
not to mention generating a CTE field to begin with!

			Ned

Date: Tue, 22 Sep 1998 11:31:47 -0700 (PDT)
From: John Mani <[email-masked]>
Reply-To: John Mani <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
To: [email-masked]
Cc: [email-masked], [email-masked]
Content-MD5: iTT+heYtyFirdfQ0/lmcyA==
Sender: [email-masked]

> 
> > Unfortunately HotMail sends out such messages - they look very much 
> >like MIME
> > messages with boundaries and mime headers and all, but they forgot to stick
> > in the MIME-Version header; they also use uuencode to encode binary data. 
> > (If the right folks are listening .. please fix this and spare us
> > client-writers some pain)
> 
> If they use UUENCODE, what do they put in the content-transfer-encoding
> field?
> 

x-uuencode.

From: Anthony Romano <[email-masked]>
To: John Mani <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MIME messages without MIME-Version header
Date: Tue, 22 Sep 1998 14:19:04 -0400 (Eastern Daylight Time)
Sender: [email-masked]

> How to handle MIME (-like) messages without the MIME-version header field ?

Here is a excerpt from RFC 2045 that addresses your concern.

In the absence of a MIME-Version field, a receiving mail user agent
   (whether conforming to MIME requirements or not) may optionally
   choose to interpret the body of the message according to local
   conventions.

----------------------
Anthony Romano
[email-masked]

Date: Tue, 22 Sep 1998 11:16:39 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: MIME messages without MIME-Version header
To: John Mani <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> How to handle MIME (-like) messages without the MIME-version header field ?

An agent is free to handle such a messae in whatever fashion it likes. I would
recommend treating it as a MIME message if the content-type and
content-transfer-encoding make sense as MIME fields, but that's just my opinion
-- your mileage may vary.

> What do IMAP servers typically do in this case ? I believe the UW server
> considers it non-mime and returns it as a singlepart message (not sure).

That's a perfectly acceptable thing to do.

> Unfortunately HotMail sends out such messages - they look very much like MIME
> messages with boundaries and mime headers and all, but they forgot to stick
> in the MIME-Version header; they also use uuencode to encode binary data. Yuk.
> (If the right folks are listening .. please fix this and spare us
> client-writers some pain)

If they use UUENCODE, what do they put in the content-transfer-encoding
field?

				Ned

Date: Tue, 22 Sep 1998 10:55:42 -0700 (PDT)
From: John Mani <[email-masked]>
Reply-To: John Mani <[email-masked]>
Subject: MIME messages without MIME-Version header
To: [email-masked]
Cc: [email-masked]
Content-MD5: RbLjF3RcSxcjLuBlDXTxyA==
Sender: [email-masked]

How to handle MIME (-like) messages without the MIME-version header field ?

What do IMAP servers typically do in this case ? I believe the UW server
considers it non-mime and returns it as a singlepart message (not sure).

Unfortunately HotMail sends out such messages - they look very much like MIME
messages with boundaries and mime headers and all, but they forgot to stick
in the MIME-Version header; they also use uuencode to encode binary data. Yuk.
(If the right folks are listening .. please fix this and spare us 
client-writers some pain)

-john

Date: Tue, 22 Sep 1998 10:14:51 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: LZJU90 compression example(s)
To: Martin Ramsch <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> BTW, has there been considerations to use a "Content-Encoding" header
> additional to the CTE?

In theory this is definitely the way to go. The problem is it doesn't
work in practice.

Current MIME agents look at content-type and content-transfer-encoding
and, if they understand what they see there, think they can remove
the transfer encoding and do something useful with the result. If you
added content-encoding, however, existing agents won't know to check it,
and will end up delivering piles of compressed binary trash to their
viewers. This is unacceptable.

This is why we defined MIME-Version -- we expected to be able to bump it when
introducing an incompatible change like this. But MIME-Version ended up botched
for political reasons, so it is no longer an option. (It turns out also to be
at the wrong level -- a technical botch, but not an insuperable one.)

So basically we're stuck with extending existing fields to express concepts
which I agree are largely (although not completely) orthogonal. Fortunately I
don't think this will prove to be much of a problem in practice, as long
as we can agree on what to do.

				Ned

Date: Mon, 21 Sep 1998 17:38:13 -0400 (EDT)
From: Nathaniel Borenstein <[email-masked]>
Reply-To: Nathaniel Borenstein <[email-masked]>
To: Ned Freed <[email-masked]>, Rick Troth <[email-masked]>
Subject: Re: LZJU90 compression example(s)
CC: Chris Newman <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

Excerpts from bulk.interesting: 21-Sep-98 Re: LZJU90 compression exam..
Rick [email-masked] (2241*)

> > > 	"Memory is cheap;  bandwidth is cheaper"
> >  
> > With all due respect to Nathaniel, he got this one wrong.   ... 
>  
> 	Yahh ... that's what I thought too when I first heard it. 
> And then,  I wasn't sure if he was speaking tongue-in-cheek or some such. 
> Eventually I decided it had something to do with economics,  programmer 
> cost,  and annoying things like that.   So I only use it here as a prod. 

For the record:  this was indeed said at least 50% tongue-in-cheek.  I
was working for Bellcore at the time, and our official mandate was to
figure out applications that would suck down the "limitless" bandwidth
that future networks would provide.  This mandate conflicted --as you
might imagine -- with many people's sense of reality, which led me to
invent the above mantra as a way of encapsulating the mindset in which
certain new applications were being proposed.

It was also said almost 8 years ago.  If I were saying it today, I would
almost certainly have said, "Bandwidth is cheap; memory is cheaper."  
It would still be tongue-in-cheek, but now it would be aimed more at the
size of Microsoft applications than at bandwidth-intensive applications.
 -- Nathaniel
--------
Nathaniel Borenstein <[email-masked]> 
http://guppylake.com/~nsb/nsb-faq.html

Date: Mon, 21 Sep 1998 16:06:11 -0500 (CDT)
From: Rick Troth <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Chris Newman <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>
Subject: Re: LZJU90 compression example(s)
Sender: [email-masked]

> > 	"Memory is cheap;  bandwidth is cheaper"
>  
> With all due respect to Nathaniel, he got this one wrong.   ... 
 
	Yahh ... that's what I thought too when I first heard it. 
And then,  I wasn't sure if he was speaking tongue-in-cheek or some such. 
Eventually I decided it had something to do with economics,  programmer 
cost,  and annoying things like that.   So I only use it here as a prod. 
 
	;-)    I truly hate the budget limits I get stuck in. 
 
> I suggest you try it before making such assumptions. 
 
	Fair enough. 
 
> I also have to run on a
> wide range of platforms, and while I've not used zlib in production (yet), 
> I have used it and found it to be very portable indeed.
 
	I don't want us to make the same mistake that was made with 
the web stuff in its adolescence.   A lot of the focus was on libwww 
rather than on the protocol.   It is completely legitimate to advocate 
a widely available, highly portable code base,  but it is wrong 
to design new protocols based on any single code base.   Right? 
 
	To be honest,  I'm not aware of any protocol problems that 
resulted from standardization on libwww.   But the goal is interoperability, 
not compatibility.   Another quote:  "rough concensus and running code" 
-- Dave Clark    Unlike code sets that talk are better than like code 
sets whether they talk or not. 
 
> 				Ned
 
	But ... back to Nathaniel's quote ... 
 
	I really am inclined to NOT advocate compression in MIME. 
Just as we might wrap SMTP in SSL,  so I'd prefer to layer the 
compression on-top-of the transaction.   So even though bandwidth is 
not as cheap as NSB's cute quote implies,  it may be wise to think of it 
as cheap when augmenting MIME and deal with the need for compression 
at a different level. 
 
	This is NOT to say that we should  "put off" 
addressing the problem.   Just that we should divi-up the work. 
Think of bandwidth as cheap in the context of the mail message. 
Think of it as cheap insofar as  >not worrying about<  "oh my, 
we need to cope with such-and-such current technology"  to the end 
that we add another vestigial feature. 
 
-- 
Rick Troth at La Casita, Houston, Texas, USA 
"When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl." 

Date: Mon, 21 Sep 1998 13:24:41 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: LZJU90 compression example(s)
To: Rick Troth <[email-masked]>
Cc: Ned Freed <[email-masked]>, Chris Newman <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

> 	"Memory is cheap;  bandwidth is cheaper"
> 		-- Nathaniel Borenstein
 
With all due respect to Nathaniel, he got this one wrong. He's not alone; many
other people, most notably the designers of the X Windows system, also got it
wrong.

> 	Say what?   CPU is cheaper than bandwidth?   Fine.

Darn tootin it is.

> But,  like Chris said,  programmers are far more expensive than either.
> So whatever you do,  go for interoperability and simplicity.
 
Simplicity can be measured in many ways. A compression scheme for which there
is no freely available highly optimized production quality version is very
complex indeed by my measure, a compression scheme for which there is a freely
available production quality version which has been ported to dozens if not
hundreds of platforms is vastly simpler.

> 	I'd be inclined to ask for GZIP with Base 64 because I can
> plug a lot of lego blocks together at the higher level.   This would
> probably wind up as a  Content-Type: application/gzip  bundle,  which
> I don't particularly care for,  but it saves on coding costs.

You're assuming a connection between compression and content type that
does not, in fact, exist. There is absolutely no reason why gzip (or
deflate) cannot be used as a CTE.

> You
> scare me when you suggest  zlib,  because that's C that might give me
> compiler/porting problems.   (I run on a W-I-D-E range of platforms.)
 
I suggest you try it before making such assumptions. I also have to run on a
wide range of platforms, and while I've not used zlib in production (yet), I
have used it and found it to be very portable indeed.

				Ned

Date: Mon, 21 Sep 1998 14:54:28 -0500 (CDT)
From: Rick Troth <[email-masked]>
To: Ned Freed <[email-masked]>
cc: Chris Newman <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>
Subject: Re: LZJU90 compression example(s)
Sender: [email-masked]

	"Memory is cheap;  bandwidth is cheaper" 
		-- Nathaniel Borenstein 
 
	Say what?   CPU is cheaper than bandwidth?   Fine. 
But,  like Chris said,  programmers are far more expensive than either. 
So whatever you do,  go for interoperability and simplicity. 
 
	I'd be inclined to ask for GZIP with Base 64 because I can 
plug a lot of lego blocks together at the higher level.   This would 
probably wind up as a  Content-Type: application/gzip  bundle,  which 
I don't particularly care for,  but it saves on coding costs.   You 
scare me when you suggest  zlib,  because that's C that might give me 
compiler/porting problems.   (I run on a W-I-D-E range of platforms.) 
 
	What are we going to do to  "get it right"  when the 
budget is against us? 
 
-- 
Rick Troth at La Casita, Houston, Texas, USA 
"When cryptography is outlawed, bayl bhgynjf jvyy unir cevinpl." 

Date: Mon, 21 Sep 1998 12:29:47 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: LZJU90 compression example(s)
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: Ned Freed <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

On Mon, 21 Sep 1998, Ned Freed wrote:
> > I think we should consider LZJU90 as a potential alternative to gzip +
> > base64.  LZJU90 is certainly much simpler than gzip, although it also
> > provides significantly less compression.  From what I can tell, both
> > LZJU90 and gzip are based on LZ77, so the main difference is that gzip
> > adds huffman coding with optional adaptive tables on top of what LZJU90
> > does and that gzip is far more widely deployed.
> 
> I'm afraid I have a big problem with this. First of all, all the debate  about
> compression speed seems to me to be beside the point. CPUs get faster all the
> time and there's no indication this will change any time soon. Networks, on the
> other hand, haven't increased in speed in a comparable fashion and there's no
> indication this is going to change either, the promise of xDSL notwithstanding.

I agree that performance of the compression algorithm isn't an issue. 
However, code complexity is an issue and LZJU90 is much simpler than
deflate (I observe that available skilled programmer and troubleshooting
cycles are perhaps even more of a scarce resource than network bandwidth
-- one of the key reasons I prefer text protocols to binary).  Those
huffman-encoded huffman tables in deflate make it rather complex (albeit
quite clever).

I don't think LZJU90 should be dismissed out of hand just because
gzip/deflate provides better compression.  This is a classic tradeoff
between complexity and network badwidth usage and deserves serious
consideration.

I'm leaning towards a gzip+4-in-5 solution on the grounds that there is a
widely available well-debugged/tested implementation (zlib).  Another
consideration is that I'd want the good MUA of the future to include an
inline PNG viewer and using gzip/deflate would permit re-use of the
compression code.  It's also my understanding that the only compression
algorithm which has been implemented in conjunction with TLS is
gzip/deflate.  This amount of code-reuse is very attractive.  But somebody
has to write the spec first and I don't have time.

However, there's a lot to be said for a solution which is simple enough
that there can be sample source in the spec.

		- Chris

Date: Mon, 21 Sep 1998 10:22:58 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: LZJU90 compression example(s)
To: Chris Newman <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

> FWIW, I think having a compressing transfer-encoding would be useful,
> especially if it's rolled out at the same time as a "widetext" toplevel
> type where compression would be particularly useful (especially if we
> decide it'd be cleaner to send around pure UCS-4 rather than UTF-16).

I agree.

> Now it's true that deploying such a transfer-encoding for general use in
> email is unlikely without something like mailcap.

I agree with this as well.

> It's also true that applying compression to already compressed objects
> (e.g., mpeg, jpeg, etc) can cause an increase in size.  The solution is
> not to use compression on such objects.

Seems reasonable enough to me.

> I think we should consider LZJU90 as a potential alternative to gzip +
> base64.  LZJU90 is certainly much simpler than gzip, although it also
> provides significantly less compression.  From what I can tell, both
> LZJU90 and gzip are based on LZ77, so the main difference is that gzip
> adds huffman coding with optional adaptive tables on top of what LZJU90
> does and that gzip is far more widely deployed.

I'm afraid I have a big problem with this. First of all, all the debate  about
compression speed seems to me to be beside the point. CPUs get faster all the
time and there's no indication this will change any time soon. Networks, on the
other hand, haven't increased in speed in a comparable fashion and there's no
indication this is going to change either, the promise of xDSL notwithstanding.

As such, it seems to me that we should go after just as much compression as we
can get without spending an excessive amount of time. And given this deflate is
the clear hands-down winner over LZJU90.

Another problem I have with the current debate is this notion that it isn't
fair to compare the sample implementation in the draft with various other
things. We have ample experience in this area, more than enough to know that
whatever you publish in an RFC is what is going to get used in 99% of the
implementations out there. As such, if you want to claim that better
performance in terms of speed or compression can be had, put it in the version
in the document or I'm not interested in hearing about it. Alternately, don't
provide any implementation in the draft and instead provide a pointer to
a high quality reference implementation like zlib.

Indeed, I'd say that this last -- a high quality reference implementation, is
an absolute requirement. The notion that MUA implementors are going to sit
around and figure out how to write better, more optimized LZJU90 encoders and
decoders simply isn't realistic.

> A few specific problems with the current LZJU90 draft:

> >               * LZJU90 <name>

> It's a layering violation for a content transfer-encoding to include a
> filename.  Filenames belong in the Content-Disposition header.

Agreed. It also fails to meet internationalization requirements, which makes it
a total nonstarter.

> The
> count makes me a bit uncomfortable.  I'm also inclined to say that CRCs
> have little value at the applications-level these days.  I can't
> say that I've ever seen a MIME object damaged in transit unintentially.
> If we're going to do any sort of integrity protection, it should be
> cryptographic, IMHO.  CRCs just don't provide enough functionality for the
> code complexity.

The bigger problem with this is that it isn't compatible with existing
decoders. People are going to want to reuse existing decoders to get to the
compressed content.

> >EBCDIC.  The 64 six-bit strings 000000 through 111111 are represented
> >by the characters "+", "-", "0" to "9", "A" to "Z", and "a" to "z".

> This is the wrong base64 alphabet.  Use the same one that MIME uses, or go
> for something more compact like base 85 (which makes sense if one is
> trying to improve compression).

Agreed -- this is another total nonstarter. We already have far too many of
these alphabets floating around. If you want a 3 in 4, use base64 and give up
on the filename and related stuff. If you want a 4 in 5, use one of the
existing ones like BTOA. BTOA has the unforunate problem of having at least one
variant with a filename at the top. But there is a BTOA variant that doesn't,
so that's the one I'd use.

				Ned

Date: Mon, 21 Sep 1998 10:03:12 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: LZJU90 compression example(s)
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: IETF-822 <[email-masked]>
Cc: MIME <[email-masked]>
Sender: [email-masked]

FWIW, I think having a compressing transfer-encoding would be useful,
especially if it's rolled out at the same time as a "widetext" toplevel
type where compression would be particularly useful (especially if we
decide it'd be cleaner to send around pure UCS-4 rather than UTF-16). 

Now it's true that deploying such a transfer-encoding for general use in
email is unlikely without something like mailcap.

It's also true that applying compression to already compressed objects
(e.g., mpeg, jpeg, etc) can cause an increase in size.  The solution is
not to use compression on such objects. 

I think we should consider LZJU90 as a potential alternative to gzip +
base64.  LZJU90 is certainly much simpler than gzip, although it also
provides significantly less compression.  From what I can tell, both
LZJU90 and gzip are based on LZ77, so the main difference is that gzip
adds huffman coding with optional adaptive tables on top of what LZJU90
does and that gzip is far more widely deployed.

A few specific problems with the current LZJU90 draft:

>               * LZJU90 <name>

It's a layering violation for a content transfer-encoding to include a
filename.  Filenames belong in the Content-Disposition header.  The
count makes me a bit uncomfortable.  I'm also inclined to say that CRCs
have little value at the applications-level these days.  I can't
say that I've ever seen a MIME object damaged in transit unintentially.
If we're going to do any sort of integrity protection, it should be
cryptographic, IMHO.  CRCs just don't provide enough functionality for the
code complexity.

>EBCDIC.  The 64 six-bit strings 000000 through 111111 are represented
>by the characters "+", "-", "0" to "9", "A" to "Z", and "a" to "z".

This is the wrong base64 alphabet.  Use the same one that MIME uses, or go
for something more compact like base 85 (which makes sense if one is 
trying to improve compression).

		- Chris

To: "Al Costanzo" <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Subject: Re: LZJU90 compression example(s) 
From: [email-masked]
Date: Mon, 21 Sep 1998 11:42:55 -0400
Sender: [email-masked]

--==_Exmh_-1704376238P
Content-Type: text/plain; charset=us-ascii

On Sat, 19 Sep 1998 17:53:20 EDT, you said:
> Therefore, any comparision is not truely fair since I assume the Base64 code
> I am using has be optimized to its fullest potential by Microsoft whereas
> the LZJU90 code is not.

It is always bad to assume that production code has been optimized.

Do you have *any* reason to assume that Microsoft optimized this code?

Do you have any reason to even assume that the Microsoft code is in
fact a totally correct and bug-free Base64 encoder?  Have you stress-tested
it with various inputs? Does it Do The Right Thing in case of memory/disk
space exhaustion? (Try attaching an uncompressed GCC source tree tarball
and see what happens?)  Have you hand-checked the code for bogosity
such as un-needed debugging code inside the central loop?  Was the Microsoft
code either compiled with an optimizer that did CSE and code hoisting, or
was the code previously hand-tuned for this?

About all you've actually proven so far is that there's reason to suspect
that Microsoft has, in fact, NOT optimized this code - presumably because
they felt it was "fast enough" already, and they had other problems to deal
with, like making sure the program didn't induce a Blue Screen of Death....

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_-1704376238P
Content-Type: application/pgp-signature

--==_Exmh_-1704376238P--

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, "Martin Ramsch" <[email-masked]>
Subject: Re: LZJU90 compression example(s)
Date: Sat, 19 Sep 1998 17:53:20 -0400
Sender: [email-masked]

Martin Writes:

>On a SUN Ultra 2 running Solaris 2.6, your LZHU90 encoder in all of my
>tests was slower than mmencode V2.7 (used for base-64 encoding).  And
>this with having complied using 'gcc -O3'.  Without '-O3' it's even
>slower ...

Martin, telling the compiler to optimize the code will not do what I call
optimizing code.

The code needs to be rewritten to work faster and compress more _BUT_ this
would be done by implementors...

Therefore, any comparision is not truely fair since I assume the Base64 code
I am using has be optimized to its fullest potential by Microsoft whereas
the LZJU90 code is not.

The rewriting of the code does not interfere with the LZJU90 algorythm, ie
no optimized encoders/decoders may handle optimized encoders/decoders just
fine.

>So your statement isn't worth much, if you don't say which base64
>encoder you used for comparison!

I have restated the mailer...

I thought I mentioned in a previous post that for all comparisions I would
use Microsoft Outlook Express version 4.72.3110.5.

I used the followin method to time----

In Outlook Express:

This may be timed by first attaching an object to a piece of email -begin-
the timing after you specifiy the filename in the "Save As" dialog box.

In the LZJU90 encoder:

Begin timing after you type the command line.

>
>> From current testing (so far, LZJU90 will _ALWAYS_ do better than
>> bas64 on raw data)
>>
>> This will not always hold true depending on the type of internal
>> compression an object may have.
>
>Did you read my former posting?  There was one example, where the
>LZJU90 encoding _was_ bigger that base64 _though_ it's been raw audio
>data.  So your statements above are wrong and misleading.
>

I read your post... My statements are NOT misleading, what type of audio
object did you compare?

I used LZJU90 on tease.wav.  Wave files are raw audio....

The original wav file was 40.7kb
The Base64 file was 57.6 kb
The LZJU90 file is 52.6 kb

What is misleading about this? LZJU90 encoded the object faster... (sorry I
do not have a stop watch :-)

>I'd put it this way:
>  LZJU90 applied to raw data most of the time yields smaller results
>  than base64 encoding, applied to data with internal compression
>  LZJU90 (nearly) always yields bigger results than base64 encoding.
>
This is an incorrect statement.  It is now YOU who are misleading people.

LZJU90 will _ALWAYS_ (at least I have not come across a time when it has not
[yet]) produce objects that are smaller than base64 encoded objects [not as
you say most of the time] if that object is RAW. It will always be faster.
(At least it is with Outlook Express)

It will also compress text documents better.

I have not tested it on may items that have builtin compression, you second
statement may be correct.

>> Even so, it is my firm belief it would be a worthy endeavor to
>> include LZJU90 into the MIME spec.
>
>Well, _if_ you can perform miracles and can convince many authors of
>MIME aware MUA to support LZJU90 (though I and prOPABLY THEY can't see
>any good advantages of it over "normal" compression plus base64), I'll
>be happy to use it in the few situations where it's useful ... :)

I do not think it will take a miricle Martin, I think it takes work and
effort to produce documents that contain R&D.  Alot of work went into LZJU90
and this work should not be slighted. It easy to rip something apart, very
hard to build.

Please believe me that if you take the time to read the draft, follow the
specification and re-write the c code optimized for your platform you MAY
see quite a difference. (but this would take work?)

Furthermore, this encoding is necessary to the internal structure of FS
(DATA portion of objects). The secod I-D I submitted.

I believe LZJU90 to be superiour to Base64 in many ways but a few.

Careful testing has shown that it does not work well with RealMedia video or
video/audio objects.  Base64 does a somewhat better job.

When moving an entire file system, as the FS draft defines, a transfer
encoding is needed that is Fast and does a good job on many different types
of objects.  LZJU90 achieves this goal.

LZJU90 was defined as a general files system object encoder/decoder, as
such, it may not work well with all objects -but- will work well with most
and I believe more than Base64.

I am building an example directory:

http://www.akc.com/fsa/examples

I am providing:

The orig. object, the base64 object and lzju90 object.

I believe in this way, the community will be able to inspect the work for
validity.

Regards to all,

-Al

Date: Mon, 14 Sep 1998 13:20:25 -0500
To: "Al Costanzo" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: application/fs vs. fs/*
Cc: "Nathaniel Borenstein" <[email-masked]>, "Steve Dorner" <[email-masked]>, <[email-masked]>, <[email-masked]>, "IETF-822" <[email-masked]>
Sender: [email-masked]

On 9/12/98 at 6:22 PM -0400, Al Costanzo wrote:

>Application/FS:
>
>I'd have to disagree,  The very fist draft of FS defined it FS  _Exactly_
>the way you are describing. I was told and told and told (you get the
>idea).... that Application was definitely the wrong place for the thing
>called "FS"

By whom? I cannot imagine that anyone in the MIME community would have said 
such a thing. So far, I have only heard Nathaniel say, "If that's the goal 
of the exercise -- i.e. to permit the gradual *evolution* of file system 
wrappers -- then a top-level name might make sense.  Otherwise, if we're 
really trying to standardize on a  single one for all time, I'd argue for 
application/fs", but I have yet to hear anyone say that application would 
be a bad idea.

>Here, I'll quote an RFC, 2046 to be specific:
>
>application -- some other kind of data, typically either uninterpreted
>binary data or information to be processed by an application.
>
>Welp (my word for Well with a sigh),
>
>It's not necessarily binary, it's specific (not some other kind) and most
>likely, if we get FS defined properly will _NOT_ be processed by an external
>application....

Note the word "typically" in the 2046 section you cite. There is nothing 
that prevents MIME interpreters from also interpreting the content of 
things which are subtypes of application; many mail applications in fact do 
just that. But like these other application subtypes, there is no further 
*MIME* information to be found in the middle of these FS parts. It is data 
that is dealt with outside of the context of this being a MIME message.

Let's take a look at some of the things you say later in your message:

>FS - is like a file archive _BUT_ its not a file archive for the purpose of
>archiving a file system set of objects.
>
>It is a collections of file system objects wrapped in an "FS" container.
>This container is in itself a file system object. "somename.fs"

This is increasing support for it being an application subtype. If FS *is* 
a container that is a file system object with no MIME structure in it, then 
it is something to be acted upon by a piece of code that considers the 
structure of the FS object, not the MIME structure of the message.

>Getting away from an external application is the whole concept of putting FS
>into MIME!

This is where the multipart stuff makes sense, but let's look at that below.

>Therefore, unless someone really convinces me otherwise, to rewrite the
>draft the way it originally was ........
>
>***Application/FS*** is not even going to be thought about (again)

You have yet to present any reasoning other than "it doesn't sound like an 
application subtype according to 2046." I've given you reasoning why it 
clearly is. Please give some other explanation before you dismiss this 
idea. I did not see the discussion regarding the first draft, but I'd like 
to hear other explantions that people gave to reject the application 
subtype. According to what you've said, you thought it was a good idea too, 
so I'd like to hear why you changed your mind (other than, "people said it 
a bunch").

>As for Multipart, I have heard from people who want it there Multipart/FS
>and people who say it does not belong.

Again, which people have said it doesn't belong. I'd like to hear their 
reasoning.

>In my mind, convince me otherwise, if FS items exist as a file system object
>they are in fact a media type, correct?

Sorry to repeat my point above, but if you are saying that there are 
currently existing applications outside of the mail system which interpret 
FS items as file system objects, then that justifies it as a subtype of 
application, not a top-level type.

>Now, if this media type to be defined is truly going to transport entire
>file system between computers of like and unlike types....
>
>Is it too much to ask for a new type to define this thing?

A new type itself is not a problem. But there is already a great deal of 
machinery within MIME to do this sort of thing. If we can reuse this 
machinery, it would be much better than reinventing the wheel. Again, I'd 
like to hear argument that the current machinery is insufficient for the 
task at hand.

>The reason I selected FILE/FS is that the level type FILE was going to
>contain file system objects that are not defined elsewhere in MIME or
>defined well.

Let's start with an easy example: GIF images. GIF images may be stored as 
file system objects, or may be dynamically displayed and never stored in a 
file system. There are two mechanisms in MIME to deal with GIF images. 
First, there is a Content-Type describing the type of data, which in this 
case is image/gif. Then there is a Content-Disposition, which describes how 
the data is to be dealt with, which might be as an file system object 
(i.e., attachment), or as a dynamically displayed image (i.e., inline). It 
might have file system attributes, which also appear in the 
Content-Disposition, like creation date, etc. MIME deals with these objects 
very nicely.

As a second example, take an HTML page with embedded GIF images. MIME has 
(newly defined) machinery for this: The HTML part, and its embedded GIF 
images, are put together in a MIME multipart/related object. The 
multipart/related object is an object to hold related objects that will not 
only be displayed together, but refer to each other within the context of 
the multipart MIME object. There is a sense of tree structure when needed; 
there is reference information like Content-ID and Content-Location, etc. 
But notice that each object within the multipart has a type (text/html, 
image/gif, etc.) that is represented by a MIME Content-Type, could have a 
Content-Disposition with further information about the object, and all of 
this at a MIME-interpretable level.

It seems no mental stretch at all to create a multipart container type 
which, instead of "related to eachother" semantics, or "digest of messages" 
semantics, had a "collection of file system objects" semantics. Each object 
could have it's own *MIME-interpretable* type information, it's own 
*MIME-interpretable* file system attributes, etc.

>how is an AVI file defined?
>
>A Real Media clip?  Are you going to stick all these into Multipart? (yuck)

Actually, both of those, since they are normally dealt with as single 
units, go under the video top-level type. And like other media types, their 
Content-Disposition can be specified too.

>To quote the same RFC:
>
>multipart -- data consisting of multiple entities of independent data types.
>The two types I have just mentioned are of a mixed data type but they in
>themselves are a single media type.  They are not multiple entities.
>
>Could it be, that these two types are being misused? I think so.

Nope. This is why 2046 also specifies:

   A media type of "video" indicates that the body contains a time-
   varying-picture image, possibly with color and coordinated sound.
   The term 'video' is used in its most generic sense, rather than with
   reference to any particular technology or format, and is not meant to
   preclude subtypes such as animated drawings encoded compactly.  The
   subtype "mpeg" refers to video coded according to the MPEG standard
   [MPEG].

   Note that although in general this document strongly discourages the
   mixing of multiple media in a single body, it is recognized that many
   so-called video formats include a representation for synchronized
   audio, and this is explicitly permitted for subtypes of "video".

>Ask any of the MIME RFC authors about me, I complained about MIME when it
>was introduced, I'm not complaining NOW -but- I may wonder about the MIME
>spec if its not flexible enough to handle the introduction of new types and
>things somewhat easily.

It is exactly the flexibility of the multipart top-level type that I am 
defending here. It is quite flexible enough to handle the subtype of "fs". 
MIME is certainly flexible enough to take new top-level types, but 
hopefully it should also be flexible enough to not require a new one when 
something fits very smoothly into a current top-level type.

I still see no evidence that multipart/fs is a problem.

pr
--
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated
Work: (217)337-6377 or (619)651-4478
Fax: (217)337-1980 or (619)651-1102

Date: Sun, 13 Sep 1998 10:01:09 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Transport Level Compression?
To: John C Klensin <[email-masked]>
Cc: Al Costanzo <[email-masked]>, Steve Dorner <[email-masked]>, MIME <[email-masked]>, IETF-822 <[email-masked]>, [email-masked]
Sender: [email-masked]

Sorry, that last one got away from me before I was finished with it. Here's
another try...

> So, while there may be value in a compressed C-T-E, it is
> something I'd expect to be applied when two communicating MTAs
> decide it is appropriate for us with any data that met some
> observable criteria (probably having more to do with the
> patterns of the bits than the semantics of the file).  As soon
> as you start talking about transferring precompressed objects of
> particular sorts, I think you are back to an application/
> subtype, not a C-T-E.

I'm afraid I'd have to disagree. Without getting into whether or not a
compression scheme independent of a data distribution format (and remember that
this is a separate proposal from the FS data distribution format) such as this
is a Good Thing, if you want to do one it pretty much has to be a CTE.

Why? Simply put, because nothing else works. The problem with overloading the
syntax of content type and subtype is that these fields already contain useful
information that agents need -- they tell you what the underlying data is and
how it is to be handled. Define a subtype for "compression" and you lose the
fact that the underlying data is, say, an Excel spreadsheet.

In fact there is language in RFC 2047 that specifically forbids this sort of
content type definition, for precisely this reason.

It has also been suggested that this would be best done at the transport
layer. Unfortunately this isn't the right place either, for two reasons:

(1) Compression is expensive, and you don't want to have to redo it after every
    hop. (There was also a suggestion that intermediaries would simply store
    the compressed form. I'm sorry, but this dog doesn't hunt, as email
    transfer necessary requires modification of the transferred content at
    every hop, which clashes badly with general compression schemes.)

(2) It interacts badly with end-to-end security in email. Specifically,
    in order for compression to work it has to be applied to the data before
    it is encrypted, not after. Encryption in email is best and usually
    done end-to-end, which means that once things get to the transport layer
    they are effectively incompressible.

So given that transport level isn't appropriate and neither is content level,
you're basically left with CTE as the only thing in the middle. Of course you
could invent a new middle layer that uses a new header field, but the problem
here is that existing agents won't recognize it and will decode based on CTE
only, but will end up with a compressed blob rather than the application data
they're expecting. At least a new CTE is required to be handled in the right
way by agents that don't support it.

> Reading some of your other notes, my guess --perhaps completely
> wrong-- is that your more general model/plan wants to see a
> hierarchy of the form of
>     C-T-E / content-type / content-subtype / parameters
> That is probably unfortunate, at least from the standpoint of
> those of us who see C-T-E and content-type as orthogonal, not
> hierarchical.

I don't think this is the intent at all. I think the intent is to define a
separate compression component. And yes, the problem with this is that
compression is in general tied to the type of data you're dealing with. But
this doesn't mean a general compression layer cannot be made to work in enough
cases to be genuinely useful.

				Ned

Date: Sun, 13 Sep 1998 09:49:18 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: Transport Level Compression?
To: John C Klensin <[email-masked]>
Cc: Al Costanzo <[email-masked]>, Steve Dorner <[email-masked]>, MIME <[email-masked]>, IETF-822 <[email-masked]>, [email-masked]
Sender: [email-masked]

> So, while there may be value in a compressed C-T-E, it is
> something I'd expect to be applied when two communicating MTAs
> decide it is appropriate for us with any data that met some
> observable criteria (probably having more to do with the
> patterns of the bits than the semantics of the file).  As soon
> as you start talking about transferring precompressed objects of
> particular sorts, I think you are back to an application/
> subtype, not a C-T-E.

I'm afraid I'd have to disagree. Without getting into whether or not a
compression scheme independent of a data distribution format (and remember that
this is a separate proposal from the FS data distribution format) such as this
is a Good Thing, if you want to do one it pretty much has to be a CTE.

Why? Simply put, because nothing else works. The problem with overloading the
syntax of content type and subtype is that these fields already contain useful
information that agents need -- they tell you what the underlying data is and
how it is to be handled. Define a subtype for "compression" and you lose the
fact that the underlying data is, say, an Excel spreadsheet.

In fact there is language in RFC 2047 that specifically forbids this sort of
content type definition, for precisely this reason.

It has also been suggested that this would be best done at the transport
layer. Unfortunately this isn't the right place either, for two reasons:

(1) Compression is expensive, and you don't want to have to redo it after every
    hop. (There was also a suggestion that intermediaries would simply store
    the compressed form. I'm sorry, but this dog doesn't hunt, as email
    transfer necessary requires modification of the transferred content at
    every hop.

> Reading some of your other notes, my guess --perhaps completely
> wrong-- is that your more general model/plan wants to see a
> hierarchy of the form of
>     C-T-E / content-type / content-subtype / parameters
> That is probably unfortunate, at least from the standpoint of
> those of us who see C-T-E and content-type as orthogonal, not
> hierarchical.

I don't think this is the intent at all. I think the intent is to define a
separate compression component. And yes, the problem with this is that
compression is in general tied to the type of data you're dealing with. But
this doesn't mean a general compression layer cannot be made to work and would
not be a Good Thing to have.

				Ned

Date: Sat, 12 Sep 1998 19:38:00 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: The MIME version number
To: Al Costanzo <[email-masked]>
Cc: Chris Newman <[email-masked]>, [email-masked], Harald Tveit Alvestrand <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>, [email-masked]
Sender: [email-masked]

> I can understand this... _however_ if the version number is now a useless thing
> lets get rid of it.  It's just using up a little bit of space but in how may
> mail messages?

There's a *big* difference between something that cannot be changed
and something that is useless. Many programs use MIME-Version to tell
whether or not a message is in MIME format. In effect it has become
a binary value.

And even if we could agree that this sort of check is itself useless and
hence best not done, we could do nothing to change the installed base that
does this check.

In short, MIME-Version is here to stay at 1.0.

> Just leaving it at 1.0 seems silly...

I agree it looks that way. But lots of engineering choices seem silly in the
abstract but are neverthless quite justifiable in practice.

				Ned

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Chris Newman" <[email-masked]>, <[email-masked]>, "Harald Tveit Alvestrand" <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, <[email-masked]>
Subject: Re: The MIME version number
Date: Sat, 12 Sep 1998 22:09:52 -0400
Sender: [email-masked]

Harald writes:

>the reason the MIME-version is stuck at 1.0 is that when the first update
>of the MIME spec was done, and it was suggested that the version number
>be bumped to 1.1, a certain large vendor with a MIME product "weeks from
>market Maxware, Norway

I can understand this... _however_ if the version number is now a useless
thing
lets get rid of it.  It's just using up a little bit of space but in how may
mail messages?

I'm sure the authors of MIME are able to think up a way of removing
something they put in...

If the version number is never to change its just a waste to have, if people
are using the version number in their software they should be notified not
to.

Its been five years, surely someone has thought of a way to bump it up or
off.

Just leaving it at 1.0 seems silly...

Only my opinion, for what its worth...

-Al

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Nathaniel Borenstein" <[email-masked]>, "Pete Resnick" <[email-masked]>
Cc: "Steve Dorner" <[email-masked]>, <[email-masked]>, <[email-masked]>, "IETF-822" <[email-masked]>
Subject: Re: application/fs vs. fs/* 
Date: Sat, 12 Sep 1998 18:22:54 -0400
Sender: [email-masked]

>Right. That's what fs is all about, so I think application/fs is the only
>justified type for what Al wants.
>

Pete,

Here are some points and tanks for the comments!

Application/FS:

I'd have to disagree,  The very fist draft of FS defined it FS  _Exactly_
the way you are describing. I was told and told and told (you get the
idea).... that Application was definitely the wrong place for the thing
called "FS"

Here, I'll quote an RFC, 2046 to be specific:

application -- some other kind of data, typically either uninterpreted
binary data or information to be processed by an application.

Welp (my word for Well with a sigh),

It's not necessarily binary, it's specific (not some other kind) and most
likely, if we get FS defined properly will _NOT_ be processed by an external
application....

Getting away from an external application is the whole concept of putting FS
into MIME!

Therefore, unless someone really convinces me otherwise, to rewrite the
draft the way it originally was ........

***Application/FS*** is not even going to be thought about (again)

To reiterate a few other points: (and not to sound like a broken record)

Compression:

FS - is like a file archive _BUT_ its not a file archive for the purpose of
archiving a file system set of objects.

It is a collections of file system objects wrapped in an "FS" container.
This container is in itself a file system object. "somename.fs"

This object is "somewhat compressed" super compression was not the goal
_but_ there is a way to Tweak LZJU90 to compress much more if you rewrite
the code for a platform specific mailer/application.

The code example does fair compression, we wanted to make sure it would
always compile.  If you tweak it, and someone else does not, it still
functions, no worries there...

With any compression algorithm depending on the file, type etc, LZJU90 will
do better in some cases and not as good in others.

The goal of LZJU90 was to prevent objects from becoming very large to have
enough compression to prevent this from happening but to be fast as well, we
don't want to wait for the object to be compressed....

Multipart:

As for Multipart, I have heard from people who want it there Multipart/FS
and people who say it does not belong.

In my mind, convince me otherwise, if FS items exist as a file system object
they are in fact a media type, correct?

Now, if this media type to be defined is truly going to transport entire
file system between computers of like and unlike types....

Is it too much to ask for a new type to define this thing?

Can I at least get consensus on this small point?

Nathaniel writes:
>A general container for OS-independent files is a GREAT and CHALLENGING
idea.  >Putting "fs" in the top level allows us to punt all the hard
decisions to the second
>level

I agree... The original FS spec. did not need a second level type, FS as
defined has no need of it but it may very be useful...  I am currently
contemplating this. Look closely how the spec is written now, but with
various rewrites who knows....

The reason I selected FILE/FS is that the level type FILE was going to
contain file system objects that are not defined elsewhere in MIME or
defined well.

how is an AVI file defined?

A Real Media clip?  Are you going to stick all these into Multipart? (yuck)

To quote the same RFC:

multipart -- data consisting of multiple entities of independent data types.
The two types I have just mentioned are of a mixed data type but they in
themselves are a single media type.  They are not multiple entities.

Could it be, that these two types are being misused? I think so.

Ask any of the MIME RFC authors about me, I complained about MIME when it
was introduced, I'm not complaining NOW -but- I may wonder about the MIME
spec if its not flexible enough to handle the introduction of new types and
things somewhat easily.

But,,,, if we are going to do something, let's do it right, I'm open for
suggestions but to put FS into either Application or Multipart, in my mind
anyway, is a misuse of them.

Putting AVI files and RM files in there is wrong as well (only my opinion),
unless this RFCs definition was changed or I am misunderstanding what the
RFC says.

Regards to all,

-Al

Date: Fri, 11 Sep 1998 16:33:38 -0500
To: Nathaniel Borenstein <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: application/fs vs. fs/* (was Re: Transfer Encoding for MIME)
Cc: "Al Costanzo" <[email-masked]>, "Steve Dorner" <[email-masked]>, [email-masked], [email-masked], "IETF-822" <[email-masked]>
Sender: [email-masked]

On 9/11/98 at 5:07 PM -0400, Nathaniel Borenstein wrote:

>Am I missing something, or are we spending more time arguing over the
>name of this thing than its design?

No, I don't think so.

>A general container for OS-independent files is a GREAT and CHALLENGING idea.

Right.

>Putting "fs" in the top level allows us to punt all the hard decisions to
>the second level.

No. That's where the argument is. MIME already has a great general 
container structure, i.e., the multipart top-level type. It also currently 
has a great way of representing things in that container structure which 
are files, i.e., the Content-Disposition with its assorted parameters. 
What's missing is a profile of the top-level multipart type which is 
specific to file system containers. That's where multipart/fs would be 
reasonable; it's a profile of our already existing general container for 
OS-independent *files*, just like multipart/digest is a container for 
messages.

What the proposed fs top-level type serves to do is introduce yet another 
general container type into MIME, only this time instead of being fully 
general, it can only hold files instead of other things, and it has no 
other internal relation to MIME structure.

>If that's the goal of the exercise -- i.e. to permit the gradual
>*evolution* of file system wrappers -- then a top-level name might make
>sense.  Otherwise, if we're really trying to standardize on a  single
>one for all time, I'd argue for application/fs.

Right. That's what fs is all about, so I think application/fs is the only 
justified type for what Al wants.

pr
--
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated
Work: (217)337-6377 or (619)651-4478
Fax: (217)337-1980 or (619)651-1102

Date: Fri, 11 Sep 1998 17:07:44 -0400 (EDT)
From: Nathaniel Borenstein <[email-masked]>
Reply-To: Nathaniel Borenstein <[email-masked]>
To: "Al Costanzo" <[email-masked]>, Pete Resnick <[email-masked]>
Subject: application/fs vs. fs/* (was Re: Transfer Encoding for MIME)
CC: "Steve Dorner" <[email-masked]>, [email-masked], [email-masked], "IETF-822" <[email-masked]>
Sender: [email-masked]

Excerpts from bulk.interesting.lists: 11-Sep-98 Re: Transfer Encoding
for MIME Pete [email-masked] (3121*)

> Fine. All that put together argues that FS is actually a container format 
> itself with multiple objects inside of it, similar to tar or StuffIt. If 
> that's the case, then there is no question it should be a subtype of 
> application: It should be application/fs. There is still no explanation 
> here of why it should be a top-level type. Would there be any reason for it 
> *not* to be a subtype of application?

Am I missing something, or are we spending more time arguing over the
name of this thing than its design?  A general container for
OS-independent files is a GREAT and CHALLENGING idea.  Putting "fs" in
the top level allows us to punt all the hard decisions to the second
level.  If that's the goal of the exercise -- i.e. to permit the gradual
*evolution* of file system wrappers -- then a top-level name might make
sense.  Otherwise, if we're really trying to standardize on a  single
one for all time, I'd argue for application/fs.

One could have had a very similar argument about "image" a few years
ago.  The fundamental guess, when the original 7 MIME types were chosen,
was that image formats would continue to evolve over the long term, so
multiple subtypes would be useful, in particular to further that
evolution.  Is this analogous to what we believe to be true about file
container formats?  -- Nathaniel
--------
Nathaniel Borenstein <[email-masked]> 
http://guppylake.com/~nsb/nsb-faq.html

Date: Fri, 11 Sep 1998 15:14:23 -0500
To: "Al Costanzo" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: Transfer Encoding for MIME
Cc: "Steve Dorner" <[email-masked]>, <[email-masked]>, <[email-masked]>, "IETF-822" <[email-masked]>
Sender: [email-masked]

On 9/10/98 at 8:29 PM -0400, Al Costanzo wrote:

>+I disagree, LZJU90 in most cases compresses objects further and is never
>harmful.

Give us a few examples. (Most of us are too lazy to do this work 
ourselves.) Take a standard JPEG. Encode it using base64. Then encode it 
using LZJU90. Tell us what the size difference is. Give a few examples like 
this (MPEG, zipped files, some sound compressions). It should be easy 
enough for you to get some files and post some numbers since you've already 
got the code running somewhere.

If it doesn't cause increases in file sizes and gets some savings, then it 
starts to be more likeable.

>>- First of all, I fail to follow the argument about why this needs to be a
>>top-level type.  Huh?
>
>Well, the FS draft which you see is actually the third attempt to define
>FS as a media type object. Feel free to help out here...
>
>The first, which Ned saw, attempted to define it as a top level type called
>FS with subtypes being file systems.
>
>The problem with that iteration of the draft (which was pointed out to me)
>would be that implementors would most likely use the type definition to help
>decode an object for a specific platform. [You'll have to trust me for a
>little while on this statement.]
>
>This would cause a problem since FS should be able to deal with this
>internally.  The concept is nothing like gzip which uses a value to tell you
>from what file system an object came from.  FS written for a specific
>platform will handle this properly.

Fine. All that put together argues that FS is actually a container format 
itself with multiple objects inside of it, similar to tar or StuffIt. If 
that's the case, then there is no question it should be a subtype of 
application: It should be application/fs. There is still no explanation 
here of why it should be a top-level type. Would there be any reason for it 
*not* to be a subtype of application?

>>I agree that it would be nice to have a way to transport
>>filesystems, but I think we would be better off working on something like
>>multipart/fs along with some new Content-Disposition parameters or
>>something.

Right. This would lend itself well to current MIME implementations.

>I am open to suggestions -but- I would like backwards compatibility and fs
>objects recognized as some form of media type. I believe what you just
>stated will not allow this to happen so I must oppose it.

You're correct; I don't think there is any way to get backward 
compatibility with current FS implementations without either (a) making it 
an application subtype and forgetting about MIME structure information or 
(b) changing FS to make it MIME like. That is, either you end up with 
something that isn't internally MIME, or something that isn't internally 
FS. I don't see any way out of that.

Like Steve, I like the idea of a MIME multipart/fs, but I don't think Al's 
proposal gets us near that. FS is just another archive format like tar or 
StuffIt.

pr
--
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated
Work: (217)337-6377 or (619)651-4478
Fax: (217)337-1980 or (619)651-1102

Date: Fri, 11 Sep 1998 10:25:37 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: Laurence Lundblade <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

On Thu, 10 Sep 1998, Laurence Lundblade wrote:
> That would be in SMTP, POP and IMAP, right? The later two are important 
> to end users because it will save them disk space and download time.

In fact, the TLS layer can include compression.  Thus one could just do
STARTTLS to get compression in all these protocols.  There are specs
nearing last call to do this.  And when a security layer is active,
lower-level compression systems (e.g., modems) cease to function, so the
compression layer has to be at or above the security layer.

> I worry about IMAP because I think I remember a long time ago Crispin 
> saying it was hard to do binary with the extent IMAP protocol syntax.

Quite the opposite, in fact.  It's much easier to do pure binary using
IMAP syntax than it is using POP or SMTP syntax.  I've heard the idea of a
"remove quoted-printable/base64 prior to transmission" extension to IMAP
tossed around several times.

		- Chris

Date: Fri, 11 Sep 1998 10:15:18 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Archive/Filesystem formats (was Re: New I-D Transfer Encoding ...)
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: IETF-822 <[email-masked]>
Cc: MIME <[email-masked]>
Sender: [email-masked]

On Wed, 9 Sep 1998, Steve Dorner wrote:
> - The draft makes an assertion that using file/fs will promote 
> interoperability between platforms.  However, all the data typing, 
> canonicalization rules, etc, that we now do with MIME are not used.  In 
> fact, all the interopability onus seems to be put on the receiving system. 
> Eg, I send you a Macintosh filesystem, one of the files has Macintosh type 
> 'ttro'.  From this, a receiving UNIX (eg) mailer is evidently supposed to 
> intuit that this is a text file and apply text canonicalization on 
> unpacking.  Seems hard.

I think this is a critical point.

We already have 3 widely deployed archive formats: tar.gz, pkzip and
stuffit.  I know there are free decoders for all three formats on Windows
and MacOS, and there are free decoders for at least the first two on Unix. 

So what is gained by introducing a new archive format?

If the new media type simply copies the facilities of tar.gz, pkzip and
stuffit, then it will *reduce* the interoperability in the installed base
by introducing a new rarely-supported format.

The only way to gain with a new archive format is to design it to be cross
platform from start to finish, rather than encapsulating platform-specific
facilities which the existing formats do just fine.  Incidentally, MIME
has already done much of this work for MIME body parts -- the most
important part of the problem being content-type. 

		- Chris

Date: Fri, 11 Sep 1998 09:46:27 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: The MIME version number
To: Harald Tveit Alvestrand <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> I believe this debacle was one of the reasons why HTTP/1.0 came out with
> very clear descriptions of how to behave when the version number changes,
> but for MIME, it was too late - we agreed that it was a constant.

> The problem with version numbers is that they assume a linear path of
> feature upgrades - 1.0 < 1.1 < 2.0, in some sense.
> If you have the "compressed CTE extension" and the "efficient multipart
> extension" (example!), and some do one and some the other, we don't have
> a linear path, but a tree, and numbers aren't too useful for labelling that.

An excellent example of where this was tried and failed is PostScript. The
label "level 2", which collects a large number of changes and extensions "under
one roof", has ended up being largely meaningless and causes all sorts of
interoperability problems in practice.

More generally, protocol extensions are fine when:

(0) You are willing to allow damage to the installed base in terms of loss of
    backwards compatibility.

(1) You are dealing with a point to point protocol and have a way of negotiating
    features. 

(2) You have a means of discovering in a store-and-forward environment what the
    final recipient's capabilities are.

We've long since agreed that (0) is a nonstarter for email. And (1) does not
apply and cannot be made to apply. Which leaves (2), which is what the
MAILCAP work is going to try and do.

> Besides, you *still* don't know what the recipient supports before you
> send the message.

Exactly.

				Ned

Date: Fri, 11 Sep 1998 10:39:54 +0200
To: "Al Costanzo" <[email-masked]>, "Chris Newman" <[email-masked]>, <[email-masked]>
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: The MIME version number
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, <[email-masked]>
Sender: [email-masked]

Al,

the reason the MIME-version is stuck at 1.0 is that when the first update
of the MIME spec was done, and it was suggested that the version number
be bumped to 1.1, a certain large vendor with a MIME product "weeks from
market"
realized in horror that his product would not accept messages labelled
with MIME-version 1.1.

I believe this debacle was one of the reasons why HTTP/1.0 came out with
very clear descriptions of how to behave when the version number changes,
but for MIME, it was too late - we agreed that it was a constant.

The problem with version numbers is that they assume a linear path of
feature upgrades - 1.0 < 1.1 < 2.0, in some sense.
If you have the "compressed CTE extension" and the "efficient multipart
extension" (example!), and some do one and some the other, we don't have
a linear path, but a tree, and numbers aren't too useful for labelling that.

Besides, you *still* don't know what the recipient supports before you
send the message.

                          Harald

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Fri, 11 Sep 1998 02:29:05 -0400 (Eastern Daylight Time)
From: John C Klensin <[email-masked]>
Subject: Re: Transport Level Compression?
To: Al Costanzo <[email-masked]>
Cc: Steve Dorner <[email-masked]>, MIME <[email-masked]>, IETF-822 <[email-masked]>, [email-masked]
Sender: [email-masked]

On Fri, 11 Sep 1998 00:23:42 -0400 Al Costanzo <[email-masked]> 
wrote:

> I want to make sure I did not misunderstand you....
> 
> The Transport Layer?  Like the Transport layer of TCP/IP?

No.  Mail transport, as in SMTP.

> I'm a software distributor, I distribute updates of my software via email.
> All the software is LZJU90 compressed and stored on a disk. when someone
> orders an update it I email them a precompressed LZJU90 object.
> 
> The objects were compressed once and may be mailed many times, why waste cpu
> cycles re-compressing the thing?

You have just described, IMO, a distribution data format, not a 
C-T-E.   Certainly, one could use a C-T-E to make unpacking a 
little more convenient but, if you were _storing_ the software 
in compressed form, you would need a rather tricky MUA and/or a 
rather fancy file system, to notice that the material was in 
that form and should be sent with this rather specialized 
content-type disguised as a C-T-E, rather than just treating it 
as binary data and using base64.

That was, of course, part of the perceived problem with 
RFC1505.  MIME separates the concept of the data type of an 
object ("content-type") from the concept of an encoding used to 
move it safely across the mail transport system.  If one 
believes that distinction is useful, 1505 hopelessly muddled it.

So, while there may be value in a compressed C-T-E, it is 
something I'd expect to be applied when two communicating MTAs 
decide it is appropriate for us with any data that met some 
observable criteria (probably having more to do with the 
patterns of the bits than the semantics of the file).  As soon 
as you start talking about transferring precompressed objects of 
particular sorts, I think you are back to an application/ 
subtype, not a C-T-E.

Note also that, absent a "mailcap"-like capability that can work 
successfully down the length of an unknown relay chain (I think 
Turing might be impressed with that, but that is another 
problem), use of a new C-T-E for this purpose probably involves 
the risk that some gateway would need to downgrade to Base64 
--exactly the conversion/deconversion situation you are trying 
to avoid.  Use of a content-type with base64 of course avoids 
that problem.

Reading some of your other notes, my guess --perhaps completely 
wrong-- is that your more general model/plan wants to see a 
hierarchy of the form of
    C-T-E / content-type / content-subtype / parameters
That is probably unfortunate, at least from the standpoint of 
those of us who see C-T-E and content-type as orthogonal, not 
hierarchical.  But, to intelligently go down that path very far, 
I suspect we will need to see more of the picture, not just one 
or two terminal leaves of the tree with promises of grand 
concept to come.

   john

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Nathaniel Borenstein" <[email-masked]>, "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, "Steve Dorner" <[email-masked]>, <[email-masked]>
Subject: C-T-E compression
Date: Fri, 11 Sep 1998 00:37:28 -0400
Sender: [email-masked]

>I tend to agree with Steve on this.  I've long thought that compressed
>C-T-E might be useful, but I feel like events have undermined this
>suspicion.  Text is generally too small to be not worth the bother of
>compressing (nor the bother of phasing in a new C-T-E), and larger media
>generally have built-in compression.  So my guess at this point is that
>phasing in this new C-T-E would be a huge amount of effort for a very
>small efficiency gain.

What do you consider huge effort?

I have tested LZJU90 with media objects that have some built-in compression.

LZJU90 does a better job than a base64 or a uuencoded object -but- again
that is not the major thrust of LZJU90.... Future work is based on the
communities acceptance or rejection of this tranfer encoding....
>
>Having said that, extending mailcap-like mechanisms to advertise
>recipient capabilities is more generally useful, and if it were
>*already* in place, phasing in the new C-T-E would be a lot less
>painful.  So I'd suggest focusing on the extension mechanisms first and
>then, if still desired, *using* them for the new C-T-E deployment.  --
>Nathaniel

It would sadden me if mailcap work interfered/delayed introduction of this
transfer encoding or others into MIME.

Accept it / reject it on its own merit not because mailcap functionality is
not where you want it to be -yet-

Only my humle little opinion...

-AL

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "John C Klensin" <[email-masked]>, "Steve Dorner" <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, <[email-masked]>
Subject: Transport Level Compression?
Date: Fri, 11 Sep 1998 00:23:42 -0400
Sender: [email-masked]

>And, of course, if there really is anything significant to be
>gained, there is a case to be made for doing the compression at
>the real transport level, rather than as a MIME C-T-E.  One
>could even imagine a transport level compression procedure
>sending binary stream compressed stuff, thereby avoiding
>sacrificing some of the efficiency gained from compression to
>the need to send ASCII.  E.g., one might have something like...
>
> client: EHLO my.dom.ain
> server: 250-BDAT-LZmm
> s:      250 EXPN
>  [...]
> client: BDAT LZmm NNN
> s: 354 expecting NNN octets
> c: [...data...]
> s: 250 Received ok
>

John,

I want to make sure I did not misunderstand you....

The Transport Layer?  Like the Transport layer of TCP/IP?

Geesh... maybe I'm working too hard but that makes no sense to me.

Correct me if I'm wrong here, but if you do it at the transport layer every
time you mail an object you have re-compress the darn thing....

Now this may make sense if your mailing an object once _BUT_ let me give you
at least one example of why you WANT to do it in the APPLICATION layer as a
transfer encoding.

I'm a software distributor, I distribute updates of my software via email.
All the software is LZJU90 compressed and stored on a disk. when someone
orders an update it I email them a precompressed LZJU90 object.

The objects were compressed once and may be mailed many times, why waste cpu
cycles re-compressing the thing?

Doing it in the application layer give you more flexibility in this
regard...

Only my opinion, but it makes a lot more sense to me....

-al

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Antony Bowesman" <[email-masked]>
Cc: "Steve Dorner" <[email-masked]>, <[email-masked]>, <[email-masked]>, "IETF-822" <[email-masked]>
Subject: Re: Transfer Encoding for MIME
Date: Thu, 10 Sep 1998 23:41:54 -0400
Sender: [email-masked]

>
>If you are looking for bacward compatibility for standalone en/de coders
>why not just register application/fs and tunnel an existing fs object
>through MIME.  The whole fs object can be contained in a part and can
>still be cut/pasted for decoding by a standalone decoder.  With the
>reverse the encoded FS object is simply added as an attachment.
>

Iteration two of this draft tried to define FS as "application/FS". As I
reread the
MIME spec. it became clear that FS really did not fit properly in this
definition.

>> >- Why isn't this multipart/fs?  Lots of good things would come from
using
>> >multipart.  I get the feeling the author has some reason he doesn't want
to
>> >use the multipart construct, but the draft doesn't say what it is.
>>
>> How right you are!  I thought I mentioned this in the draft?
>
>Is it only a backward compatibility issue for non MIME mailers.  This
>doesn't sound like a reason to invent a new top level type.

It's more than what I mentioned, basically I'm pondering a method of
registering
all file system objects in the namespace FILE, the FS draft you see skims
the
top of a deeper concept for future work.  The concept is still in its birth
stage, it
has been rewitten quite a few times.

There are many objects that are currently not defined in MIME, if you going
to do work that deals with all the objects that define a FILE system, it
would seem to be prudent to begin discussing a methodology to register all
these types.

LZJU90 and FS are just two drafts, hopefully there will be others that build
upon this work, truthfully the definition of the top level type will most
likely be an entire I-D by itself but I want to put the concept out to the
community now, the simple definition in FS draft was a way to circulate the
concept and see how it was received.

>> >fact, all the interopability onus seems to be put on the receiving
system.
>> >Eg, I send you a Macintosh filesystem, one of the files has Macintosh
type
>> >'ttro'.  From this, a receiving UNIX (eg) mailer is evidently supposed
to
>> >intuit that this is a text file and apply text canonicalization on
>> >unpacking.  Seems hard.
>
>The onus IS always on the receiving system to be more flexible in what
>it accepts than it generates.  The same issues would apply to the UNIX
>mailer if it received a multipart/appledouble, unless or course the
>sending mailer were to generate a suitable multipart/alternative
>suitable for all recipients.  (Using up lots of bandwidth in the
>process!)
>
>> >- I am skeptical that the type can adequately represent current and
future
>> >filesystems.  In current form, I don't see how it can represent critical
>> >items from the Macintosh filesystem (for example, the Macintosh filetype
I
>
>And many other bits of information available in the resource fork.
>
>>
>> >  4.3 mandates using ".fs" on the end of filenames; why?
>> >
>> Why do Zip files end in .zip text files end in .txt, etc.  Since FS
objects
>> are actually a collection of file system objects that may be stored as a
>> file.  It needs a file hook
>> designation for registration purposes like on Windows NT/98/95, ".fs"
seemed
>> to be a logical choice. Don't you agree?
>
>Another reason against the top level, if you require a hook it shows
>that you are looking for an external function to decode the type.
>Therefore why not application/fs

This was not my intent. It was only so that a standard naming convention
would be used.

The concept behind FS is to get away from the concept that an external
function would be used.  The desire is that the mailers user interface would
have the fs software built into it to handle all of this but windows
NT/95/98 and others need a file hook to know they need to activate the
mailer software to deal with this type of object.

Just like an .htm or .html file invoke the internet explorer when these
files are laid down on your desktop.
>
>> >I agree that it would be nice to have a way to transport
>> >filesystems, but I think we would be better off working on something
like
>> >multipart/fs along with some new Content-Disposition parameters or
>> >something.
>
>I agree that using nested multiparts is a better way of going forward.
>Certain mailers already implement the ability to attach directory
>structures as nested multiparts.  It is the directory/file structure
>information that is missing.  There are a number of CD attributes in
>Steve Dorner's (+others) RFC2183 that match what you propose for the new
>entity and these would seem better suited to enhance a multipart/fs.
>These could be extended to include some of the attributes you propose.
>

I'll read the RFC....

Thanks for your advise!

-AL

Date: Thu, 10 Sep 1998 19:57:28 -0700 (PDT)
From: Laurence Lundblade <[email-masked]>
To: John C Klensin <[email-masked]>
cc: Steve Dorner <[email-masked]>, IETF-822 <[email-masked]>, MIME <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME 
Sender: [email-masked]

That would be in SMTP, POP and IMAP, right? The later two are important 
to end users because it will save them disk space and download time.

I worry about IMAP because I think I remember a long time ago Crispin 
saying it was hard to do binary with the extent IMAP protocol syntax.

I like the transport layer because the compression can be negotiated so
you don't have the mailcap / deployment problem. Further compression may
be of interest on small devices communicating over slow links (e.g.
wireless). On some of these devices memory is limited enough that some
algorithms have to have their hufman or other encoding tables throttled.
For example gzip uses a 32Kb table, and getting an implementation onto a
Palm Pilot would not be straight forward. Transport layer compression
would allow the encoding table size to be negotiated. 

I also like the ability to avoid the 15% or so overhead needed to send ASCII.

I don't have hard numbers handy, but I have done some tests compressing
mail messages, even small ones and concluded it was well worth it. Also
I'm not sure that link layer compression can come close to the
effectiveness of doint it at higher layers, even for general purpose
compression algorithms. Link layer compression schemes cannot operate on
more than a few hundred bytes at a time or they will introduce latency
that is highly undesirable and unexpected. 

Perhaps the big downside to transport layer compression is that it has to
be redone with each transport session there by sucking up lots of cycles
on the Internet's mail servers (that might be better used to crack DES:-). 

LL

On 10 Sep 1998, John C Klensin wrote:
> ....
>  client: EHLO my.dom.ain
>  server: 250-BDAT-LZmm
>  s:      250 EXPN
>   [...]
>  client: BDAT LZmm NNN
>  s: 354 expecting NNN octets
>  c: [...data...]
>  s: 250 Received ok
> 
> Now, like Steve, I'm not completely convinced any of this is 
> worth it.  But, if one is going to do it, then it may be worth 
> the slight extra trouble (and perhaps easier deployment) to do 
> it at transport and get all of the gains possible.
> 
>     john
> 
> 

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Chris Newman" <[email-masked]>, "Harald Tveit Alvestrand" <[email-masked]>, <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, <[email-masked]>
Subject: The MIME version number
Date: Thu, 10 Sep 1998 23:11:16 -0400
Sender: [email-masked]

>
>The MAILCAP stuff is about making the third alternative possible.
>
>                 Harald A
>

Harald,

Regarding your thoughts on this -- I have been contemplating what you said
as well as statements others have made; and, in a message to me, Patrik made
the following comment:

>....
>So far, all clients have to handle all transport encodings (Base64 and QP)
>to be MIME compliant. Should this new one also be mandatory (not!), and if
>so, what is the fallback procedure for a client that gets an unknown
>transport encoding?
>

In light of what I'm hearing, I would like to discuss another possibility.

I was at the IETF meeting in 1993 when it was stated that the MIME version
number shall always be 1.0.

Why? I don't know, I did not ask.

I do think that it is possible that this version number may/could be used
for something useful.

Usually version numbers in software release follow a specific pattern, but
there is no law that states that this pattern need be followed.

My suggestion is that the version number be used to determine what features
the MIME mailer in question understands.  They do NOT have to follow 1,2,3,4
etc. but they could.

Version . Sub

Sub could be used to determine this mailer understands transfer-encodings
x.y.z etc.

Version could be used to determine feature sets this version of the mailer
supports.

In this way, all you need to do is scan the version number for the
features/encodings ( and of course the first step is to get all the MIME
compliant mailers to do the scanning first.

In my mind it seems like a waste of chars to always put version 1.0 there.

May be a radical idea, but it seems like common sense to me....

Regards to all,

-al

Date: Fri, 11 Sep 1998 11:55:59 +0900
From: Antony Bowesman <[email-masked]>
To: Al Costanzo <[email-masked]>
CC: Steve Dorner <[email-masked]>, [email-masked], [email-masked], IETF-822 <[email-masked]>
Subject: Re: Transfer Encoding for MIME
Sender: [email-masked]

Al,

If you are looking for bacward compatibility for standalone en/de coders
why not just register application/fs and tunnel an existing fs object
through MIME.  The whole fs object can be contained in a part and can
still be cut/pasted for decoding by a standalone decoder.  With the
reverse the encoded FS object is simply added as an attachment.

> >- Why isn't this multipart/fs?  Lots of good things would come from using
> >multipart.  I get the feeling the author has some reason he doesn't want to
> >use the multipart construct, but the draft doesn't say what it is.
> 
> How right you are!  I thought I mentioned this in the draft?

Is it only a backward compatibility issue for non MIME mailers.  This
doesn't sound like a reason to invent a new top level type.

> >fact, all the interopability onus seems to be put on the receiving system.
> >Eg, I send you a Macintosh filesystem, one of the files has Macintosh type
> >'ttro'.  From this, a receiving UNIX (eg) mailer is evidently supposed to
> >intuit that this is a text file and apply text canonicalization on
> >unpacking.  Seems hard.

The onus IS always on the receiving system to be more flexible in what
it accepts than it generates.  The same issues would apply to the UNIX
mailer if it received a multipart/appledouble, unless or course the
sending mailer were to generate a suitable multipart/alternative
suitable for all recipients.  (Using up lots of bandwidth in the
process!)

> >- I am skeptical that the type can adequately represent current and future
> >filesystems.  In current form, I don't see how it can represent critical
> >items from the Macintosh filesystem (for example, the Macintosh filetype I

And many other bits of information available in the resource fork.

> 
> >  4.3 mandates using ".fs" on the end of filenames; why?
> >
> Why do Zip files end in .zip text files end in .txt, etc.  Since FS objects
> are actually a collection of file system objects that may be stored as a
> file.  It needs a file hook
> designation for registration purposes like on Windows NT/98/95, ".fs" seemed
> to be a logical choice. Don't you agree?

Another reason against the top level, if you require a hook it shows
that you are looking for an external function to decode the type. 
Therefore why not application/fs

> >I agree that it would be nice to have a way to transport
> >filesystems, but I think we would be better off working on something like
> >multipart/fs along with some new Content-Disposition parameters or
> >something.

I agree that using nested multiparts is a better way of going forward. 
Certain mailers already implement the ability to attach directory
structures as nested multiparts.  It is the directory/file structure
information that is missing.  There are a number of CD attributes in
Steve Dorner's (+others) RFC2183 that match what you propose for the new
entity and these would seem better suited to enhance a multipart/fs. 
These could be extended to include some of the attributes you propose.

It is better to create something that more easily fits within MIME (i.e.
existing parsers SHOULD treat unknown multipart types as
multipart/mixed) and would provide implementors a simpler development
route.  Backward compatibility can be achieved as above.

> I have had this discussion over and over.  I read the MIME specs. In my
> mind, FS objects are media types they are not encodings. They are however
> encoded.
> AND so are Zip files and JPEG's and Real Media and ARJ  files ....

They are media types but you would not expect a MIME implementation
written on a particular platform to understand the intricacies of an
existing FS object containing a VMS stucture, instead you would want to
hand that off to something that does.

Rgds

-- 
Antony Bowesman
[email-masked]

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Ned Freed" <[email-masked]>, "Steve Dorner" <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Date: Thu, 10 Sep 1998 20:35:59 -0400
Sender: [email-masked]

...
>So all you need to do, really, is have a single VMS-specific parameter
called,
>say, vms-fdl, to store this string in, and you've solved the problem of
sending
>VMS files around with their attributes intact.
>
...

Your points have been received, at the appropriate time the FS spec will
need to be updated accordingly.  Your help will be appreciated.

-al

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Steve Dorner" <[email-masked]>, <[email-masked]>, <[email-masked]>, "IETF-822" <[email-masked]>
Subject: Re: Transfer Encoding for MIME
Date: Thu, 10 Sep 1998 20:29:54 -0400
Sender: [email-masked]

Hello Steve,

Thanks for the comments...

>I don't find a lot to like in either proposal.
>
>Compression:
>
>- Most really big data types have integral compression (sound, video,
>images, etc) and applying another compression algorithm to them is
>pointless or harmful.

+I disagree, LZJU90 in most cases compresses objects further and is never
harmful.

>
>- Even when you can get compression (such as with text), I'm unconvinced it
>buys you much.  There's so much low-layer compression going on.  I would
>like to see numbers indicating a solid benefit from application layer
>compression in email, a benefit worth the deployment penalty.
>
++ I have given you source code crunch all the numbers you want :-)

but seriously, LZJU90 has pretty good compression, I (we) have compared it
to binary objects that were encoded in other formats.  LZJU90 does a better
job than most.

Keep in mind compression was NOT the major goal of LZJU90.....

++ The compression we were interested in was to prevent an object from
becomming much larger than the file system's objects initial size.  LZJU90
achieves this goal and more.   It is also quite fast.  If the program was
written for a specific platform you would see a speed increase and somehat
better compression.

Remember the example in the draft has been written for clarity and
portability not speed and the best compression possible.

++ Although mentioned in one of the drafts, it may be worthwile for me to
mention that both LZJU90 & FS were designed to work with any mailer that was
able to support the UTF-8 character set and complied with RFC-822.

I will mention more about this later...

>FILE/FS:
>
>- First of all, I fail to follow the argument about why this needs to be a
>top-level type.  Huh?

Well, the FS draft which you see is actually the third attempt to define
FS as a media type object. Feel free to help out here...

The first, which Ned saw, attempted to define it as a top level type called
FS with subtypes being file systems.

The problem with that iteration of the draft (which was pointed out to me)
would be that implementors would most likely use the type definition to help
decode an object for a specific platform. [You'll have to trust me for a
little while on this statement.]

This would cause a problem since FS should be able to deal with this
internally.  The concept is nothing like gzip which uses a value to tell you
from what file system an object came from.  FS written for a specific
platform will handle this properly.

Now the MIME spec says a top level type needs a subtype, since there is now
no subtype

Instead of complaining about the type/subtype requirement, I reworded the
section to read FILE/FS (for now)

Please remeber this is only the first public viewing of the FS draft...
>
>- Why isn't this multipart/fs?  Lots of good things would come from using
>multipart.  I get the feeling the author has some reason he doesn't want to
>use the multipart construct, but the draft doesn't say what it is.

How right you are!  I thought I mentioned this in the draft?

You could take an FS media object, open it up in MS WordPad or some other
text editor, 'select all' and then paste the object into a mail message on a
mail program that is RFC 822 compliant (that knows utf-8) and send it on its
way....  If the receiver had a
stand alone FS encoder/decoder they could just feed the object in and the
object would be successfully decoded...   If an FS object was split between
parts with MIME control elements in between elements, the FS object would
not be interoperalble with none MIME mailers. Why care? I explain elsewhere
in this email.
>
>- The draft makes an assertion that using file/fs will promote
>interoperability between platforms.  However, all the data typing,
>canonicalization rules, etc, that we now do with MIME are not used.

There are a number of issues here.  First and foremost I would like to see
both
LZJU90 and FS on standards track and in the MIME spec.

Please bear in mind that these protocols are currently in use, I am not sure
how widespread.

In my mind, allowing backward interoperability with existing users and
allowing MIME
to use it are important to me.

You must consider that no matter what position I take on this issue, someone
will surely complain to me about my position and about me to others (I feel
loved :-).

MIME diehards will say forget the non-MIME compliant mailers,  use the new
robust MIME feature set(s) that MIME has to offer. [you]

While non-MIMEer's will tell me that I'm abadoning them for the MIME world
and leaving them in the cold.

Those that implemented RFC 1505 will ask if they wasted their implementing
RFC 1505 which we the authors of 1505 saw as a lightweight alternative to
MIME.

The answers:

Since FS and LZJU90 are able to function on any mail system that support the
utf-8 char set and RFC 822 , the two drafts were written to allow possible
inter-operability.

This interoperability might not be the slickest, a little cut and paste but
it will work.

So I have elected to select a path that would hopefully not aggrevate
everyone, my success remains to be seen :-)

>fact, all the interopability onus seems to be put on the receiving system.
>Eg, I send you a Macintosh filesystem, one of the files has Macintosh type
>'ttro'.  From this, a receiving UNIX (eg) mailer is evidently supposed to
>intuit that this is a text file and apply text canonicalization on
>unpacking.  Seems hard.

Your assement is correct on how FS is to work is correct.  It does work
though.
I will need to look at the VMS portion again and will rely on help from the
community...

>
>- I am skeptical that the type can adequately represent current and future
>filesystems.  In current form, I don't see how it can represent critical
>items from the Macintosh filesystem (for example, the Macintosh filetype I
>mention above. I'd be interested to see what Ned has to say regarding VMS
>here.)  Perhaps specific problems of current filesystems could be coped
>with by adding to this draft, but it seems to me that for future growth
>we'd wind up needing to reinvent a lot of the registry and standards
>process we already have, for use inside this thing.
>
>- Perhaps this is just a first-draft problem, but there are various minor
>strangenesses.  Section 4.6 seems to forbid using message/partial to
>transport file/fs objects; why?

+Answer: nteroperabilty with stand alone FS encoders/decoders and other
mailers.

4.7 mandates that applications use CD:
>attachment; why?

+Answer:
I'm not an expert MIME users. The MIME mailer I use is MS Outlook Express.
I have noticed that when CD is set to attachment, it allows the user a
choice of saving the object to disk or to open the object.

A user may want to store an FS object without decoding it or may want to
decode it immediately, this setting seemed to perform this funtionality...
Correct me if I'm wrong.

This is the was FS object should be handled.

>  4.3 mandates using ".fs" on the end of filenames; why?
>
Answer:

Why do Zip files end in .zip text files end in .txt, etc.  Since FS objects
are actually a collection of file system objects that may be stored as a
file.  It needs a file hook
designation for registration purposes like on Windows NT/98/95, ".fs" seemed
to be a logical choice. Don't you agree?

>It seems to me that the proposal throws away lots of good work that's been
>done with MIME.

+Answer:

I hope not! This was not my intent at all!!! and I don't see what makes you
think this.

>I agree that it would be nice to have a way to transport
>filesystems, but I think we would be better off working on something like
>multipart/fs along with some new Content-Disposition parameters or
>something.

I am open to suggestions -but- I would like backwards compatibility and fs
objects recognized as some form of media type. I believe what you just
stated will not allow this to happen so I must oppose it.

I have had this discussion over and over.  I read the MIME specs. In my
mind, FS objects are media types they are not encodings. They are however
encoded.
AND so are Zip files and JPEG's and Real Media and ARJ  files ....

-Al

Date: Thu, 10 Sep 1998 17:03:16 +0200
To: [email-masked]
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME 
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Sender: [email-masked]

At 10:47 10.09.98 -0400, [email-masked] wrote:
>Some of us blinked.  Can you post a reference/URL/summary of the current
>state of the MAILCAP work?
>
I'm almost as much in the dark as you...the AGENDA is at

ftp://ftp.ietf.org/ietf/98aug/mailcap-agenda-98aug.txt

but I haven't seen minutes, so I was asking for someone else to sum up...

           Harald

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Thu, 10 Sep 1998 8:35:21 -0700
From: Dan Wing <[email-masked]>
To: [email-masked]
CC: [email-masked], [email-masked], [email-masked]
Subject: Re: New I-D Transfer Encoding for MIME 
Sender: [email-masked]

>> The MAILCAP stuff is about making the third alternative possible.
>
>Harald:
>
>Some of us blinked.  Can you post a reference/URL/summary of the current
>state of the MAILCAP work?

I didn't take detailed meeting minutes, but what I have is:

  This WG will focus on how to determine the capabilities of an
  email recipient.  Things such as public PGP or S/MIME keys,
  supported content types (PowerPoint, PDF, WordPerfect), sound,
  and other information.  This information could be used to generate
  a message that will be usable by the recipient.   Many problems
  were discussed briefly or offline, such as a recipient that uses 
  multiple access methods to their mailbox (pager, cellular phone, 
  Eudora, Netscape, etc.).

DNS was tossed around as a solution, but this is perhaps an over-abuse
of DNS.

Ned Freed was volunteered as the chair.  He may have been trying to
protest, but I couldn't hear it over the many people saying his name
when a volunteer was being requested.  :-)

-Dan Wing

To: Harald Tveit Alvestrand <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME 
From: [email-masked]
Date: Thu, 10 Sep 1998 10:47:40 -0400
Sender: [email-masked]

--==_Exmh_-371015376P
Content-Type: text/plain; charset=us-ascii

On Thu, 10 Sep 1998 09:07:54 +0200, Harald Tveit Alvestrand said:
> - Perform techno-magical invocations to find out whether he supports it or
> not.
> 
> The MAILCAP stuff is about making the third alternative possible.

Harald:

Some of us blinked.  Can you post a reference/URL/summary of the current
state of the MAILCAP work?

/Valdis

--==_Exmh_-371015376P
Content-Type: application/pgp-signature

--==_Exmh_-371015376P--

Date: Thu, 10 Sep 1998 09:34:50 -0400 (EDT)
From: Nathaniel Borenstein <[email-masked]>
Reply-To: Nathaniel Borenstein <[email-masked]>
To: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>, Steve Dorner <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Sender: [email-masked]

I tend to agree with Steve on this.  I've long thought that compressed
C-T-E might be useful, but I feel like events have undermined this
suspicion.  Text is generally too small to be not worth the bother of
compressing (nor the bother of phasing in a new C-T-E), and larger media
generally have built-in compression.  So my guess at this point is that
phasing in this new C-T-E would be a huge amount of effort for a very
small efficiency gain.

Having said that, extending mailcap-like mechanisms to advertise
recipient capabilities is more generally useful, and if it were
*already* in place, phasing in the new C-T-E would be a lot less
painful.  So I'd suggest focusing on the extension mechanisms first and
then, if still desired, *using* them for the new C-T-E deployment.  --
Nathaniel
--------
Nathaniel Borenstein <[email-masked]> 
http://guppylake.com/~nsb/nsb-faq.html
--------
Nathaniel Borenstein <[email-masked]> 
http://guppylake.com/~nsb/nsb-faq.html

Date: Thu, 10 Sep 1998 09:07:54 +0200
To: "Al Costanzo" <[email-masked]>, "Chris Newman" <[email-masked]>
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Sender: [email-masked]

At 20:02 09.09.00 -0400, Al Costanzo wrote:
>
>Hello Harald,
>
>>>
>>.....which brings us straight back to the MAILCAP debate about how to get
>>more info about a mail recipient than his address.
>
>I starting to feel like I'm missing the point. Why is this needed for
>LZJU90?
>
>I see no need....

It's kind of a tangent, but an important one.

As Chris Newman said, deploying a new content-transfer-encoding WILL lead
to situations where the sender does not know whether the recipient can
handle the new content-transfer-encoding or not.
The alternatives are, roughly:

- Send in an old c-t-e, in which case you have no benefit
- Ask people, and keep your own list of who handles what
  (possibly by sending in the new c-t-e and fielding the enraged complaints)
- Perform techno-magical invocations to find out whether he supports it or
not.

The MAILCAP stuff is about making the third alternative possible.

                 Harald A

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Wed, 09 Sep 1998 23:26:10 -0700 (PDT)
From: Ned Freed <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
To: Steve Dorner <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

> - I am skeptical that the type can adequately represent current and future
> filesystems.  In current form, I don't see how it can represent critical
> items from the Macintosh filesystem (for example, the Macintosh filetype I
> mention above. I'd be interested to see what Ned has to say regarding VMS
> here.)  Perhaps specific problems of current filesystems could be coped
> with by adding to this draft, but it seems to me that for future growth
> we'd wind up needing to reinvent a lot of the registry and standards
> process we already have, for use inside this thing.

I talked briefly with Al about this privately and meant to make some additional
comments, but then along came the Chicago IETF and I got, well, sidetracked.
But now is as good a time as any...

There are two issues here: Directory structure and file attributes.

VMS doesn't have the concept of a filesystem within which devices are mounted.
It's the other way around: There are devices at the top level, which may or may
not contain filesystems. There are also pseudo-devices that can be used to
elevate some point in the filesystem hierarchy on a given device to being
another device.

In practice this is a nonissue, however, as people don't move around the
contents of multiple devices. And when you ignore this the semantics of VMS
filesystem actually become a proper subset of those of, say, UNIX. (However,
this does include things like hard links.)

File characteristics are another matter. The current draft doesn't do this
correctly. But it turns out that this is actually a pretty simple problem to
solve for VMS -- there's this thing called the file description language, or
FDL, that can be used to construct a single string that contains all of the
attributes of a file that aren't actually part of the file's data. (BTW, most
VMS filesystems use an on-disk format called ODS-2, which can only accomodate a
single fork per file. But there's a VMS format called Spiralog that can
accomodate multiple forks the way NT systems can.) This includes stuff like
file organization, record size, record type, version limit, global buffer
count, protection, and so on. All told there are about 25 different things you
can put in an FDL string that correspond to stuff that's not actually in the
file data.

FDL strings are quite similar to the application/applefile part of a
multipart/appledouble structure. But the nice thing is that every VMS system
comes fully loaded with routines to generate and process FDL strings. It's a
native structure, it's fully extensible, backwards compatible, the OS people
update it as necessary,  and so on. In fact we've been sending them around as a
MIME content-type parameters for years now, which half a dozen different
products from different vendors interoperating very nicely in handling VMS
files via email.

So all you need to do, really, is have a single VMS-specific parameter called,
say, vms-fdl, to store this string in, and you've solved the problem of sending
VMS files around with their attributes intact.

But if this means that every filesystem is going to need its own string
for its own set of file attributes, it does sort of beg the question of
how cross-filesystem mapping will work, both for existing and future
filesystems.

> It seems to me that the proposal throws away lots of good work that's been
> done with MIME.  I agree that it would be nice to have a way to transport
> filesystems, but I think we would be better off working on something like
> multipart/fs along with some new Content-Disposition parameters or
> something.

I think there is general agreement that interoperable transport of filesystems
would be a good, perhaps even great, thing. The question is how...

				Ned

Date: Thu, 10 Sep 1998 03:01:47 -0400 (Eastern Daylight Time)
From: John C Klensin <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
To: Steve Dorner <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

On Wed, 09 Sep 1998 22:41:29 -0500 Steve Dorner 
<[email-masked]> wrote:

> I don't find a lot to like in either proposal.
> 
> Compression:
>... 
> - Even when you can get compression (such as with text), I'm unconvinced it 
> buys you much.  There's so much low-layer compression going on.  I would 
> like to see numbers indicating a solid benefit from application layer 
> compression in email, a benefit worth the deployment penalty.

And, of course, if there really is anything significant to be 
gained, there is a case to be made for doing the compression at 
the real transport level, rather than as a MIME C-T-E.  One 
could even imagine a transport level compression procedure 
sending binary stream compressed stuff, thereby avoiding 
sacrificing some of the efficiency gained from compression to 
the need to send ASCII.  E.g., one might have something like...

 client: EHLO my.dom.ain
 server: 250-BDAT-LZmm
 s:      250 EXPN
  [...]
 client: BDAT LZmm NNN
 s: 354 expecting NNN octets
 c: [...data...]
 s: 250 Received ok

Now, like Steve, I'm not completely convinced any of this is 
worth it.  But, if one is going to do it, then it may be worth 
the slight extra trouble (and perhaps easier deployment) to do 
it at transport and get all of the gains possible.

    john

Date: Wed, 9 Sep 1998 22:41:29 -0500
To: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
From: Steve Dorner <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Sender: [email-masked]

I don't find a lot to like in either proposal.

Compression:

- Most really big data types have integral compression (sound, video, 
images, etc) and applying another compression algorithm to them is 
pointless or harmful.

- Even when you can get compression (such as with text), I'm unconvinced it 
buys you much.  There's so much low-layer compression going on.  I would 
like to see numbers indicating a solid benefit from application layer 
compression in email, a benefit worth the deployment penalty.

FILE/FS:

- First of all, I fail to follow the argument about why this needs to be a 
top-level type.  Huh?

- Why isn't this multipart/fs?  Lots of good things would come from using 
multipart.  I get the feeling the author has some reason he doesn't want to 
use the multipart construct, but the draft doesn't say what it is.

- The draft makes an assertion that using file/fs will promote 
interoperability between platforms.  However, all the data typing, 
canonicalization rules, etc, that we now do with MIME are not used.  In 
fact, all the interopability onus seems to be put on the receiving system. 
Eg, I send you a Macintosh filesystem, one of the files has Macintosh type 
'ttro'.  From this, a receiving UNIX (eg) mailer is evidently supposed to 
intuit that this is a text file and apply text canonicalization on 
unpacking.  Seems hard.

- I am skeptical that the type can adequately represent current and future 
filesystems.  In current form, I don't see how it can represent critical 
items from the Macintosh filesystem (for example, the Macintosh filetype I 
mention above. I'd be interested to see what Ned has to say regarding VMS 
here.)  Perhaps specific problems of current filesystems could be coped 
with by adding to this draft, but it seems to me that for future growth 
we'd wind up needing to reinvent a lot of the registry and standards 
process we already have, for use inside this thing.

- Perhaps this is just a first-draft problem, but there are various minor 
strangenesses.  Section 4.6 seems to forbid using message/partial to 
transport file/fs objects; why?  4.7 mandates that applications use CD: 
attachment; why?  4.3 mandates using ".fs" on the end of filenames; why?

It seems to me that the proposal throws away lots of good work that's been 
done with MIME.  I agree that it would be nice to have a way to transport 
filesystems, but I think we would be better off working on something like 
multipart/fs along with some new Content-Disposition parameters or 
something.

Date: Thu, 10 Sep 1998 09:25:48 +0800
To: Pete Resnick <[email-masked]>
From: Dave Crocker <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Cc: "Al Costanzo" <[email-masked]>, "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Sender: [email-masked]

At 08:06 PM 9/9/98 -0500, Pete Resnick wrote:
>I think this is a concern, but not a showstopper for LZJU90 itself.

I think that Al's confusion is appropriate.  Yes, use of this type of new
feature needs to be co-ordinated, but that's true for almost ANY new
feature.  And this particular new feature might be so appealing that it
forces to get even more serious about providing a good capabilities
exchange/announcement mechanism.  

However I believe that work is entirely separate from this particular
proposal.  Let's just use the current proposal as added impetus for
pursuing the mailcap work aggressively.

d/

=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
Dave Crocker                                         Tel: +1(408)246 8253
<mailto:[email-masked]>                        675 Spruce Drive
                                                  Sunnyvale, CA 94086 USA
Brandenburg Consulting
<http://www.brandenburg.com>                         Tel: +60(19)3299 445
                                               Post Office Box 296, U.P.M.
Fax: +1(408)246 8253                     Serdang, Selangor 43400 MALAYSIA

Date: Wed, 9 Sep 1998 20:06:14 -0500
To: "Al Costanzo" <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Cc: "Chris Newman" <[email-masked]>, "Harald Tveit Alvestrand" <[email-masked]>, "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Sender: [email-masked]

On 9/9/00 at 8:02 PM -0400, Al Costanzo wrote:

>I starting to feel like I'm missing the point. Why is this needed for
>LZJU90?
>
>I see no need....

It's not LZJU90 in particular; it's any MIME feature which you don't have a
good assurance that the other side will have implemented. In the case of a
CTE, it's especially yucky because it makes the data pretty much unreadable
if you don't know the CTE.

I think this is a concern, but not a showstopper for LZJU90 itself.

pr

[PS: Why's your clock set to 2000 instead of 1998?]
--
Pete Resnick <mailto:[email-masked]>
QUALCOMM Incorporated
Work: (217)337-6377 or (619)651-4478
Fax: (217)337-1980 or (619)651-1102

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "Chris Newman" <[email-masked]>, "Harald Tveit Alvestrand" <[email-masked]>
Cc: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Date: Sat, 9 Sep 2000 20:02:52 -0400
Sender: [email-masked]

Hello Harald,

>>
>.....which brings us straight back to the MAILCAP debate about how to get
>more info about a mail recipient than his address.
>I remember a BOF that I was just skimming past on this subject in Chicago;
>care to give us a capsule summary?
>
>(IMHO, a working mailcap has the potential to dramatically change the
>playing field of the email world wrt deployment of new features; can that
>be part of why it's so slow to get anywhere....?)
>
>                    Harald

I starting to feel like I'm missing the point. Why is this needed for
LZJU90?

I see no need....

-Al

Date: Wed, 09 Sep 1998 23:08:14 +0200
To: Chris Newman <[email-masked]>, Al Costanzo <[email-masked]>
From: Harald Tveit Alvestrand <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

At 12:29 09.09.98 -0700, Chris Newman wrote:
>I support the general idea of a compressing CTE, although the rules for
>when it is permitted have to be carefully drawn up so that it doesn't
>create compatibility problems (in particular, it MUST NOT be used without
>some form of prior agreement between the sender and recipient).
>
.....which brings us straight back to the MAILCAP debate about how to get
more info about a mail recipient than his address.
I remember a BOF that I was just skimming past on this subject in Chicago;
care to give us a capsule summary?

(IMHO, a working mailcap has the potential to dramatically change the
playing field of the email world wrt deployment of new features; can that
be part of why it's so slow to get anywhere....?)

                    Harald

-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

Date: Wed, 09 Sep 1998 12:29:36 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: New I-D Transfer Encoding for MIME
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: Al Costanzo <[email-masked]>
Cc: IETF-822 <[email-masked]>, MIME <[email-masked]>
Sender: [email-masked]

On Sat, 9 Sep 2000, Al Costanzo wrote:
> The first two Internet drafts I submitted are the beginning of this work.
> 
> * draft-costanzo-lzju90-mime-01.txt -
> 
> is a document that redefines LZJU90 as a new transfer encoding for MIME.
> This transfer encoding also contains compression. This is an item that was
> requested in 1993.
> 
> This is the second public iteration of the document.
> 
> LZJU90 is in the true public domain, there is no patent on it. A patent
> search was made.

I support the general idea of a compressing CTE, although the rules for
when it is permitted have to be carefully drawn up so that it doesn't
create compatibility problems (in particular, it MUST NOT be used without
some form of prior agreement between the sender and recipient).

I am curious why you're proposing LZJU90, instead of Deflate/gzip (RFC
1951).  Deflate as the distinct advantage of already being documented in
an RFC and having freely available source code.  A simple mapping can
convert deflate to gzip and take advantage of the fact that gzip decoders
are available on all platforms.

I also object to the use of a base64 alphabet different from the base64
alphabet defined in the MIME standard.  And if compression is the intent,
then perhaps something more aggressive than base64 should be applied to
the output of the compresssion operation.

		- Chris

Reply-To: "Al Costanzo" <[email-masked]>
From: "Al Costanzo" <[email-masked]>
To: "IETF-822" <[email-masked]>, "MIME" <[email-masked]>
Cc: "Leigh Sandy" <[email-masked]>
Subject: New I-D Transfer Encoding for MIME
Date: Sat, 9 Sep 2000 05:25:00 -0400
Sender: [email-masked]

9/9/1998

Greetings to all,

I am Al Costanzo one of the co-authors of RFC 1505.  Some may remember that
in the past I opposed MIME placement on standards track.

Five years later, MIME is widely deployed, I may have been wrong :-).

I am willing to admit to everyone that I was quite pig-headed back then
(maybe still am in some areas but I am working on it!).  A number of IETF
members contacted us ( the authors of RFC1505 ) and 'suggested' we redefine
a few RFC1505 items for MIME.

I for one refused the request, consulted the other co-authors and basically
did nothing in this area since the RFC was release in 1993.

Consider the above an apology for being pig-headed and not listening.

The first two Internet drafts I submitted are the beginning of this work.

* draft-costanzo-lzju90-mime-01.txt -

is a document that redefines LZJU90 as a new transfer encoding for MIME.
This transfer encoding also contains compression. This is an item that was
requested in 1993.

This is the second public iteration of the document.

LZJU90 is in the true public domain, there is no patent on it. A patent
search was
made.
---------

* draft-costanzo-fs-mime-00.txt - UNLIKE the LZJU90 draft, this draft is
preliminary, therefore please treat it as such. It is the beginning of a
directory attachment concept
for MIME.

There were many rewrites to this document. This is the first public release.

The attempt is/was to create a standardized mechanism to attach directory
structures to email that:

* does not  require multi-encodings to survive mail transport
* does not rely on modular programs (plug-ins) for functionality

Currently, people (users) either create a tar ball or use some program like
pkZip, Gzip or what-have-you to move a directory structure via email.  The
problem with this method is that both the receiver and sender MUST have the
same software (or at least the software must support the same 'encoding
scheme') to achieve the transfer properly.

Granted, many programs are freely downloadable....

However, my intent is to create/define a standard mechanism that may be
embedded into the mailer program's GUI directly.

I have had many requests for this functionality from 'users'.

I have received a bit of interest on this from implementers.

I invite the Internet community to review the documents.

You may send constructive comments to [email-masked]

Since I am very mindful of email lists, if you would like to carry on this
discussion off this list, you may subscribe to a list we have set up
just for comments on these drafts.

To subscribe send a message to the File System Attachment list:
[email-masked]

In the body of the message place the word: subscribe

The list has verification turned on, so you'll need to respond to
Majordomo with its request for verification in accordance with
the instructions it sends.

Thank You All for your kind attention in this matter.

-Al
----------------------------------------------------------------------------
----------------
And Remember --- No Matter Where You Go, There You Are...and so is God!

Date: 03 Sep 1998 09:37:00 +0200
From: [email-masked] (Kai Henningsen)
To: [email-masked]
Subject: Re: format=flowed : why not use HTML?
Organization: Organisation? Me?! Are you kidding?
Comment: Unsolicited commercial mail will incur an US$100 handling fee per received mail.
Sender: [email-masked]

[email-masked] (Jamie Zawinski)  wrote on 20.08.98 in <[email-masked]>:

> Chris Newman wrote:

> > In hindsight, we should have developed a markup language using tricks like
> > _italics_, *bold* and >excerpt for email, so that the markup language
> > itself would be plain text.  Even that would require style-run merging,
> > but it probably would have worked.  Unfortunately, it's too late to
> > introduce something like that now.
>
> Brad Templeton (I think) proposed something like that years and years
> ago, and came to the conclusion that it didn't really work.  Then later

Well, AFAICT ist's used very successfully in a local BBS network. I think  
there exist numerous clients that don't do too bad a job of parsing it.

It's not perfect, but it sure beats both nothing and HTML. *And*, since  
it's (A) so easy to create and parse manually, and (B) lots of people  
already use it manually, _it is already deployed_!

(And my current client does convert four instances in this mail into  
highlighted text (not that much it can do with DOS text mode), for  
example.)

I wonder if it would be useful to write up an informational RFC on that  
...

MfG Kai

Date: Tue, 01 Sep 1998 16:52:32 -0700 (PDT)
From: Chris Newman <[email-masked]>
Subject: Re: format=flowed : why not use HTML?
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

On Tue, 1 Sep 1998 [email-masked] wrote:
> OK.. is the *exact* problem the interaction with QP?

Quoted-printable has a very bad reputation in many parts of the field. 
The derogatory term "quoted-unreadable" is quite common.  In fact, there
is some evidence that a significant number of people prefer the
non-interoperable "just-send-8" (and hope you don't lose) to interoperable
quoted-printable.  In those parts of the Internet which hate
quoted-printable, probably the only thing more loathed is text/html. :-)

Should the fact these solutions are hated by some stop us from trying to
deploy interoperable and backwards compatible standards?  No. 

Should it make us think twice before using such services in combination
with an acceptance-sensitive proposal?  Definitely.

> Should we instead have strong language saying what
> MUST be done to support format=flowed correctly if using (for instance)
> ISO8859-4?

When 8BITMIME transport is lacking and 8-bit is used, obviously
downconversion is a MUST for interoperability.  But in all other cases,
quoted-printable would be dangerous with this particular media type.  I'd
go so far as to say that quoted-printable SHOULD NOT be used to protect
the whitespace at the end of the lines, as use of quoted-printable will
likely cause more harm than losing the trailing whitespace (and in the
latter case, the harm is more likely to be blamed on the guilty party).
Client implementors of this proposal should be strongly encouraged to
use ESMTP 8BITMIME instead of quoted-printable whenever possible (but this
need not be a normative statement).

> Unacceptable: Language that implies this is only usable with English.

You know me well enough to know I'd never propose such a thing.

		- Chris

Date: Tue, 1 Sep 1998 12:02:53 -0700
To: Laurence Lundblade <[email-masked]>, [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: format=flowed : why not use HTML?
Cc: Chris Newman <[email-masked]>, [email-masked]
Sender: [email-masked]

At 8:58 AM -0700 9/1/98, Laurence Lundblade wrote:

>"QP encoding MUST be used when necessary to encoded non-US characters;
>however it SHOULD not be used unless necessary. It is preferable to risk
>formatting damage than to insert formatting crud that may cause the
>format=flow text format to be rejected by end users as incompatible with
>plain text"

Or even

While Quoted-Printable encoding MUST be used when necessary to encode
non-US-ASCII characters, it SHOULD NOT be used unless required for such
uses; that is, if the SMTP server supports 8BITMIME, or the message
only contains US-ASCII characters, there is no need for
Quoted-Printable encoding.

The intent of Format=Flowed is to allow user agents to generate flowed
text which is non-obnoxious when viewed as pure Text/Plain; use of
Quoted-Printable hinders this and may cause Format=Flowed to be
rejected by end users.

Date: Tue, 1 Sep 1998 08:58:25 -0700
To: [email-masked]
From: Laurence Lundblade <[email-masked]>
Subject: Re: format=flowed : why not use HTML?
Cc: Chris Newman <[email-masked]>, [email-masked]
Sender: [email-masked]

I think Chris's concern is that it will get stigmatized as yet another
format that includes formatting crud that users don't want to see.  So how
about something like this.

"QP encoding MUST be used when necessary to encoded non-US characters;
however it SHOULD not be used unless necessary. It is preferable to risk
formatting damage than to insert formatting crud that may cause the
format=flow text format to be rejected by end users as incompatible with
plain text"

LL

At 10:17 AM -0400 9/1/98, [email-masked] wrote:
>On Mon, 31 Aug 1998 19:32:16 PDT, you said:
>> Before I'd be willing to support a proposed standard last call on
>> format=flowed, I'd want to make sure there's strong language discouraging
>> the use of quoted-printable with it, and that we've had a reasonable
>
>OK.. is the *exact* problem the interaction with QP?  Would dropping
>to Base64 fix it?  Should we instead have strong language saying what
>MUST be done to support format=flowed correctly if using (for instance)
>ISO8859-4?
>
>I have to rate as show-stopper any proposal that doesn't permit correct
>operation with ISO8859-x.  I'd even be tempted to say that it has to deal
>with RFC1556, except that's only "Informational".
>
>Acceptable:  List of additional 'MUST' and 'SHOULD' items for use with
>QP and/or non-ASCII text, which may result in extra hoops to jump through
>for the MUA author.
>
>Unacceptable: Language that implies this is only usable with English.
>
>--
>				Valdis Kletnieks
>				Computer Systems Senior Engineer
>				Virginia Tech
>
>
>Content-Type: application/pgp-signature
>
>Attachment converted: llundblade:Untitled 59 (????/----) (0002560B)

To: Chris Newman <[email-masked]>
Cc: [email-masked]
Subject: Re: format=flowed : why not use HTML? 
From: [email-masked]
Date: Tue, 01 Sep 1998 10:17:32 -0400
Sender: [email-masked]

--==_Exmh_456656672P
Content-Type: text/plain; charset=us-ascii

On Mon, 31 Aug 1998 19:32:16 PDT, you said:
> Before I'd be willing to support a proposed standard last call on
> format=flowed, I'd want to make sure there's strong language discouraging
> the use of quoted-printable with it, and that we've had a reasonable

OK.. is the *exact* problem the interaction with QP?  Would dropping
to Base64 fix it?  Should we instead have strong language saying what
MUST be done to support format=flowed correctly if using (for instance)
ISO8859-4?

I have to rate as show-stopper any proposal that doesn't permit correct
operation with ISO8859-x.  I'd even be tempted to say that it has to deal
with RFC1556, except that's only "Informational".

Acceptable:  List of additional 'MUST' and 'SHOULD' items for use with
QP and/or non-ASCII text, which may result in extra hoops to jump through
for the MUA author.

Unacceptable: Language that implies this is only usable with English.

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_456656672P
Content-Type: application/pgp-signature

--==_Exmh_456656672P--