Subject: Compression Order

From: "David Fischer" <[email-masked]>
Date: Tue, 5 Mar 2002 11:42:50 -0600

AS3?...We are beginning to implement the Compression Draft
specification:

http://www.ietf.org/internet-drafts/draft-ietf-ediint-compre
ssion-00.txt

and an implementation decision has arisen.  When Compression
is applied in concert with Signatures, which should be
applied first?

    Compress + Sign    or
    Sign + Compress

        and

    Compress + Sign + Encrypt   or
    Sign + Compress + Encrypt

Different companies have presented valid business cases for
each alternative.

The advocates for Signing first desire the signature to be
over readable text (you know what you are signing -- which
is why we never Sign after Encrypting).

The advocates for Compressing first desire performance
improvements.  In this case, Compression is compared as just
another encoding scheme so it is really still plain text.

Discussions thus far have resulted in a consensus that the
implementors should be able to apply these functions in
either order when sending.  When receiving, the
implementations should be able to process either case.

We are looking for further discussions. . .

Regards,

David Fischer
Drummond Group.

---

From: joe mcverry <[email-masked]>
Date: Tue, 05 Mar 2002 14:10:11 -0500

Signatures tend to be small - don't they?

Joe

> David Fischer wrote:
> 
> We are beginning to implement the Compression Draft specification:
> 
> http://www.ietf.org/internet-drafts/draft-ietf-ediint-compression-00.txt
> 
> and an implementation decision has arisen.  When Compression is
> applied in concert with Signatures, which should be applied first?
> 
>     Compress + Sign    or
>     Sign + Compress
> 
>         and
> 
> Different companies have presented valid business cases for each
> alternative.
> 
> The advocates for Signing first desire the signature to be over
> readable text (you know what you are signing -- which is why we never
> Sign after Encrypting).
> 
> The advocates for Compressing first desire performance improvements.
> In this case, Compression is compared as just another encoding scheme
> so it is really still plain text.
> 
> implementors should be able to apply these functions in either order
> when sending.  When receiving, the implementations should be able to
> process either case.
> 

-- 
----------- 
Joe McVerry
American Coders Ltd.
POBox 97462
Raleigh, NC   27624  USA
919.846.2014 (voice/fax)
http://www.americancoders.com
Home Of OBOE - an EDI and EDI/XML Translator
    and xBaseJ - xBase Database Engine For Java

---

From: "Kit (Christopher) Lueder" <[email-masked]>
Date: Tue, 05 Mar 2002 14:46:51 -0500

I think the issue is not the size of the signatures (which may be small), but
the size of the document being signed, which may be enormous. Compressing the
document prior to signing it means less processing overhead when performing the
signature operation. (This is a benefit for compressing first.)

But I lean toward the Signing First approach. Another benefit of that is you can
uncompress it and make it readable and not violate the signature. If you
compress first, uncompressing it means you are no longer looking at a signed
document.

Kit Lueder.
MITRE.

-- 
    _/    _/             Kit C. J. Lueder       
   _/   _/         _/   The MITRE Corp.         Tel:  703-883-5205
  _/_/_/    _/  _/_/_/ 7515 Colshire Dr        Cell: 703-577-2463
 _/   _/   _/    _/   Mailstop W658           FAX:  703-883-3383
_/    _/  _/    _/   McLean, VA 22102-7508   Mail: [email-masked]
Worse than an unanswered question is an unquestioned answer.

---

From: "David Fischer" <[email-masked]>
Date: Tue, 5 Mar 2002 14:20:24 -0600

Yes, signatures are small but the effort to create them is directly related to
the size of the file being signed.

Regards,

David Fischer
Drummond Group.

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of joe mcverry
Sent: Tuesday, March 05, 2002 1:10 PM
To: David Fischer
Cc: [email-masked]
Subject: Re: Compression Order

Joe

---

From: James M Galvin <[email-masked]>
Date: Tue, 05 Mar 2002 17:08:22 -0500 (EST)

I'm not aware of any technical reason (security-related or otherwise)
why you would have to choose one over the other however, semantically,
the reasoning that applies to sign before encrypting applies here also.

If you sign the compressed (encrypted) form instead of the original
"text", then strictly speaking the only security service available to
you is origin authentication.  It would be *wrong* to associate or infer
or in any way tightly-couple any authorization semantics with the
signature.  It would be *wrong* to associate or infer or in any way
tightly-couple any characteristic or attribute that suggests the entity
that created the signature had any knowledge whatsoever of the actual
content that was signed.

I would expect this to be an issue in most business applications and
contexts.

Jim

--
James M. Galvin <[email-masked]>

On Tue, 5 Mar 2002, David Fischer wrote:

    Date: Tue, 05 Mar 2002 11:42:50 -0600
    From: David Fischer <[email-masked]>
    To: [email-masked]
    Subject: Compression Order

        Compress + Sign    or
        Sign + Compress

            and

    Regards,

    David Fischer
    Drummond Group.

---

From: Kepa Zubeldia <[email-masked]>
Date: Tue, 05 Mar 2002 20:11:10 -0700

What if you need more than one signature?  Common situation in 
healthcare.  It would have to be

sign - sign - sign - compress - encrypt

So you know what you are signing at each step, before it turns into "mush".

Kepa

David Fischer wrote:

> and an implementation decision has arisen.  When Compression is applied 
> in concert with Signatures, which should be applied first?
> 
>  
> 
>     Compress + Sign    or
> 
>     Sign + Compress
> 
>  
> 
>         and
> 
>  
> 
> The advocates for Signing first desire the signature to be over readable 
> text (you know what you are signing -- which is why we never Sign after 
> Encrypting).
> 
>  
> 
> The advocates for Compressing first desire performance improvements.  In 
> this case, Compression is compared as just another encoding scheme so it 
> is really still plain text.
> 
>  
> 
> Discussions thus far have resulted in a consensus that the implementors 
> should be able to apply these functions in either order when sending.  
> When receiving, the implementations should be able to process either case.
> 
>  
>

---

From: "David Fischer" <[email-masked]>
Date: Wed, 6 Mar 2002 10:04:09 -0600

True, but we don't support multiple signatures.

Serial signatures (signing over the previous signature) are fairly simple, as a
construct.  Parallel signatures are undefined at the moment.  We could easily
support serial signatures in AS2.

A compressed signature is not "mush" any more than receiving a .ZIP attachment
would be.  OTOH, receiving an encrypted object without possessing the
private-key is truly "mush" to the recipient/potential signer.  In the case of
serial signatures and compression, the inner part can be read (decompressed)
without disturbing the signature so I don't know why multiple signatures would
be an argument either way?

Regards,

David Fischer
Drummond Group.

-----Original Message-----
From: Kepa Zubeldia [mailto:[email-masked]]
Sent: Tuesday, March 05, 2002 9:11 PM
To: David Fischer
Cc: [email-masked]
Subject: Re: Compression Order

Kepa

---

From: "Carl Hage" <[email-masked]>
Date: Fri, 8 Mar 2002 15:18:05 -0800

From:           	"David Fischer" <[email-masked]>

> http://www.ietf.org/internet-drafts/draft-ietf-ediint-compression-00.txt

The use of:

Content-Type: application/pkcs7-mime; smime-type=compressed-data

seems inappropriate.

The application "pkcs-mime" shouldn't be used. PKCS7 is a different
and inconsistent method of doing the same as what MIME does.

It really should be pure MIME, independent of PKCS7.

The original intent in the MIME design was to use Content-Transfer-
Encoding for compression. From RFC2048: "Transfer encodings can 
be used to apply general-purpose non-lossy compression algorithms to 
MIME entities.  "

The intent was for someone to define a new encoding type to include 
compression, e.g. "Content-Transfer-Encoding:gzip" or
"Content-Transfer-Encoding:base64-gzip", which could be equivalent to 
httpd content-encoding, or base64 encoded gzip.

Adding a new content-transfer-encoding would be universally useful
in all MIME messages, and simplify encoding.

Alternatively, a new type, e.g.

"Content-Type: application/mime-gzip" or 
"Content-Type: application/mime-encoded; encoding=gzip"

could be defined to mean an application that decodes, then
processes mime headers. However, it would be better to use:

"Content-Type: encoding/gzip" (already built into Netscape)

Where the media type "encoding" means decode and continue parsing
MIME headers, just like multipart or message.

Shouldn't compression be defined completely independently of 
encryption, signatures, etc.? Then EDIINT can use MIME compression 
with multipart/signed, etc. and other independent MIME standards.
--------------------------------------------------------------------------
Carl Hage                                              C. Hage Associates
<mailto:[email-masked]> Voice/Fax: 1-408-244-8410      1180 Reed Ave #51
<http://www.chage.com/chage/>                          Sunnyvale, CA 94086

---

From: "Carl Hage" <[email-masked]>
Date: Fri, 8 Mar 2002 15:18:05 -0800

The use of:

seems inappropriate.

---

From: "Greg Vesper" <[email-masked]>
Date: Mon, 11 Mar 2002 09:45:06 -0700

Well said - I agree entirely with Carl's assessment.  Binding compression at
the crypto layer is inherently more restrictive and less interoperable than
compressing at the MIME layer.

Cheers,
-Greg.

> [mailto:[email-masked]]On Behalf Of Carl Hage
> Sent: Friday, March 08, 2002 4:18 PM
> To: [email-masked]
> Subject: Re: Compression Draft
>
>
>

---

From: "David Fischer" <[email-masked]>
Date: Mon, 11 Mar 2002 21:47:44 -0600

This was our first thought too but this method was rejected by the Area
Director.

Regards,

David Fischer
Drummond Group.

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of Greg Vesper
Sent: Monday, March 11, 2002 10:45 AM
To: Carl Hage; [email-masked]
Subject: RE: Compression Draft

Cheers,
-Greg.

---

From: [email-masked]
Date: Thu, 21 Mar 2002 15:21:19 -0800 (PST)

> This was our first thought too but this method was rejected by the Area

I'm sorry, but this is flatly wrong. I never rejected the notion of adding
compression at the MIME layer. What I rejected was doing it with a separate
field nobody has ever seen before and no existing agent will know to look for.
Nor is it acceptable for this to be done by using a content-type based
wrapper.

As I pointed out before, the addition of compression to MIME is done by adding
one or more content transfer encodings. The process for defining a new CTE is
laid out in RFC 2048. The process is intentionally a stringent one, but is
isn't impossible.

Adding, say, zlib-base64 or even zlib-base85 would be easy. Adding something
like zlib-binary or zlib-8bit is harder, but still doable.

				Ned

---

From: "David Fischer" <[email-masked]>
Date: Thu, 21 Mar 2002 20:55:51 -0600

My apologies, I wasn't clear.

Since we are working with HTTP, we cannot use CTE.  Whatever we use must be the
same for SMTP and HTTP.  I remember that all the options we discussed for
binding at the MIME level were not acceptable.   Was I mistaken?

I have been searching for that thread (between Ned Freed and Terry Harding) and
I can't find it.

David Fischer
Drummond Group.

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Thursday, March 21, 2002 5:21 PM
To: David Fischer
Cc: Greg Vesper; IETF EDIINT
Subject: RE: Compression Draft

				Ned

---

From: [email-masked]
Date: Thu, 21 Mar 2002 19:08:37 -0800 (PST)

> Since we are working with HTTP, we cannot use CTE.  Whatever we use must be
> the same for SMTP and HTTP.

I fail to see any justification for this restriction. Per RFC 2616 section
19.4.5, CTE is indeed not used by HTTP, but all that means is that should you
need to gateway you have to remove the CTE.

You already have to accomodate the base64 and quoted-printable CTEs, so
you have to deal with CTEs regardless of whether or not define a new
compressed one.

> I remember that all the options we discussed for

Yes.

> I have been searching for that thread (between Ned Freed and Terry Harding) and

I don't recall the conversation.

				Ned

---

From: "Carl Hage" <[email-masked]>
Date: Thu, 21 Mar 2002 19:55:21 -0800

> Since we are working with HTTP, we cannot use CTE.  Whatever we use
> must be the same for SMTP and HTTP. 

Well, that makes sense to me, but HTTP already has a specification 
for compression, using Content-Encoding, but does not explicitly 
permit Content-Transfer-Encoding.

For email, a "Content-Transfer-Encoding: zlib-base64" (or gzip-base64)
is equivalent to encoding with the equivalent of HTTP
"Content-Encoding: gzip" then encoded with
"Content-Transfer-Encoding: base64".

If you are using HTTP, you can send binary data and compress using 
the existing "Content-Encoding: gzip" specification defined in the HTTP 
RFC. This would be equivalent to a
"Content-Transfer-Encoding:zlib-binary".

If you are using SMTP, we could define "zlib-base64" Content-Transfer-
Encoding. The software would compress (as when using content-
encoding), then base64 encode. There is no need for "zlib-binary" as 
long as compression is done at the top level.

["zlib-base64" is probably a good name since the RFC defining the   
format calls it zlib.]

Though the headers are a little different when used with HTTP or 
SMTP, the software works pretty much the same. It would be easy to 
write a MIME encoder/decoder that would work with either HTTP or 
SMTP, but adjust the MIME headers as needed.

Adding a new content-transfer-encoding format is useful outside the 
scope of EDI and is a simpler extension than the proposed alteration 
encryption/signature MIME types. It's also explicitly mentioned as the 
strategy intended in the MIME RFCs.
--------------------------------------------------------------------------
Carl Hage                                              C. Hage Associates
<mailto:[email-masked]> Voice/Fax: 1-408-244-8410      1180 Reed Ave #51
<http://www.chage.com/chage/>                          Sunnyvale, CA 94086

---

From: "Carl Hage" <[email-masked]>
Date: Thu, 21 Mar 2002 19:55:21 -0800

---

From: "Terry Harding" <[email-masked]>
Date: Fri, 22 Mar 2002 08:50:21 -0700

Carl, Ned,

Our goal in developing a compression method for AS1 and AS2 was to find
something that already existed and could be used for SMTP and HTTP(s) and
any future transport protocols defined by the EDIINT group.

For HTTP we looked at using Content-Encoding, but since this only applied to
the outermost layer of a message, we would not be able to take advantage of
the benefits of compressing the mime layers inside the message and any
compression applied on the outermost layer would not have any effect
compressing a binary blob of encrypted data.  I also looked at using
Content-Encoding within the inner mime layers, but was informed by Ned that
utilizing Content-Encoding within MIME was not acceptable.  We also
looked at utlizing CTE in the inner most layers, but also received
objections from the AS2 authors that CTE anywhere within the message was not
encouraged. Also the MIME group did not have a CTE defined for compression
and we would have to develop a brand new CTE acceptable to EDIINT players
and the MIME guardians. We also looked at the compression spec from the
S/MIME group, as this is the security mechanism utlized by AS1 and AS2.
Note: the S/MIME group had already developed a compression spec.

For SMTP we looked at using Content-Encoding, but again we were discouraged
from this approach. It was suggested that we use CTE with new encodings
defined for compression.  We also looked at compression developed by the
S/MIME group.

So it came down to utilizing a compression method already defined by a
working group  and having it approved by the EDIINT forks or developing a
new mechanism and having it approved by two working groups. The EDIINT group
on the most part trys to utilize existing methods and explain their usage in
our applicability statements.

So the reasons above led us to utilize the compression method developed by
the S/MIME group. The original EDIINT compression was first released in
September of 2001 to the ediint ietf group for comments and suggestions.  At
that time we were looking for discussion on the selected method.  We
received some suggestions and comments, but nothing close to the remark from
Carl, indicating that the compression method selected was "OVERKILL".

Note: Currently 7 AS1 vendors have implemented the selected compression
method and 5 additional
vendors will be adding compression for AS2..

Terry Harding
Cyclone Commerce

---

From: [email-masked]
Date: Fri, 22 Mar 2002 19:45:17 -0800 (PST)

> any future transport protocols defined by the EDIINT group.

I note in passing that it is extremely unlikely that EDIINT is going to be
around to develop such bindings.

> encouraged.

Then the AS2 authors are in need a reality check. CTEs within messages are not
only encouraged, they are essential. And here is nothing, repeat NOTHING, they
can do about this.

Like it or not, HTTP elected to go one way on this and SMTP elected to
go another. Both approaches have advantages and disadvantages, as you note.
But you cannot change them and you cannot make them compatible. And more to
the point, you cannot get rid of the use of CTEs in SMTP.

> Also the MIME group did not have a CTE defined for compression

Which offers a third set of tradeoffs.

> from this approach.

Actually, you were told it is flatly unacceptable on technical grounds, and
why. There's nothing capricious or arbitrary going on here.

> It was suggested that we use CTE with new encodings

> our applicability statements.

And if the WG choses S/MIME compression that's fine. The only reason I jumped
in s the situation in regards to defining new compression services in SMTP had
been, and continues to be, mischaracterized.

				Ned

---

From: "Dick Brooks" <[email-masked]>
Date: Mon, 25 Mar 2002 22:31:54 -0600

Ned,

Regarding:
>I note in passing that it is extremely unlikely that EDIINT
>is going to be around to develop such bindings.

I wish to express the importance of EDIINT AS2 by pointing out that the Gas
Industry Standards Board, an ANSI SDO, (now the North American Energy
Standards Board),ref: http://www.naesb.org/, references EDIINT AS2 in it's
Internet E-Commerce standard. The Federal Energy Regulatory Commission
(http://www.ferc.gov) within the Department of Energy
(http://www.energy.gov/) has mandated NAESB's Internet E-Commerce/AS2
standard into law and all Interstate Gas pipelines are required to use the
NAESB/AS2 standard for Internet E-Commerce.

Note: The NAESB E-Commerce standard references the portion of AS2 that
complies with RFC 2616, RFC 2617, RFC 2388 for framing and RFC 1847/2015 for
MIME identification of PGP payloads. PGP performs all compression functions
within the NAESB portion of EDIINT AS2 and is therefore not dependent on
CTE's to identify compressed content.

Please do not associate the "compression issue" with all of EDIINT AS2. I
believe the issues raised with compressed content are germane to the UCC
portion of EDIINT AS2 and do not impact the Energy Industry/NAESB "profile"
within the specification. Please contact me (co-chair of NAESB's Internet
E-Commerce standard committee) to discuss the Energy Industry portion of
EDIINT AS2. I want to ensure that the Energy Industry E-Commerce
specifications are aligned with the principles and processes defined by the
Internet Engineering Task Force.

Thank you,

Dick Brooks
Systrends, Inc
7855 South River Parkway, Suite 111
Tempe, Arizona 85284
Web: www.systrends.com <http://www.systrends.com>
Phone:480.756.6777,Mobile:205-790-1542,eFax:240-352-0714

-----Original Message-----
From: [email-masked]
[mailto:[email-masked]]On Behalf Of
[email-masked]
Sent: Friday, March 22, 2002 9:45 PM
To: Terry Harding
Cc: Carl Hage; [email-masked]
Subject: Re: Compression Draft

> objections from the AS2 authors that CTE anywhere within the message was
not

Then the AS2 authors are in need a reality check. CTEs within messages are
not
only encouraged, they are essential. And here is nothing, repeat NOTHING,
they
can do about this.

> our applicability statements.

And if the WG choses S/MIME compression that's fine. The only reason I
jumped
in s the situation in regards to defining new compression services in SMTP
had
been, and continues to be, mischaracterized.

				Ned