Subject: Re: Signed headers in email (was Re: Draft for signed headers)
From: Chris Newman <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Date: Wed, 31 Mar 1999 15:53:38 -0800 (PST)

On Fri, 26 Mar 1999, Charles Lindsey wrote:
> >(4) Any attempt at a canonicalization algorithm for mail header signing is
> >    doomed to failure from the outset.  It will be ambiguous, too complex,
> >    or inadequate.
> 
> And that is a matter you decide by examining the proposals and finding
> holes in them. Not by asserting ex cathedra that no solution exists. I
> have proposed a canonicalization algorithm which may or may not work. It
> should be examined, discussed, and improved.

I skimmed your proposal and it was both too complex and had ambiguities. 
The obvious way to simplify it is to stop worrying about email-safety. 
It's not worth pointing out the ambiguities until the algorithm is
sufficiently simple to be viable.

> Fine. So you are happy to let the news people develop this tool on their
> own, and you are not going to complain if it turns out later that they
> have omitted some small tweak that would have made it work much better in
> mail?

Yes.

> That is fine by me. But we shall in any case try to make it as
> mail-proof as we can while we are about it.

If you really want this to work in news, I suspect the KISS principle is
far more important than "mail-proof".

		- Chris

Date: Tue, 30 Mar 1999 19:03:06 -0800 (PST)
From: Chris Newman <[email-masked]>
Subject: Parsing issue in MIME spec
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Sender: [email-masked]

I noticed that the MIME specs are not particularly clear about which MIME
headers are parsed according to RFC 822's specials or MIME's tspecials.

Content-Type, Content-Disposition and any other header with MIME
parameters is obviously parsed using MIME's tspecials.

Content-ID is parsed using RFC 822's specials by virtue of the fact it is
supposed to have identical syntax to Message-ID.  In particular, this
means that "/", "?" and "=" are legal unquoted on the left-hand-side of
the "@".

"ietf-token" and "x-token" rules have unspecified syntax.  This means the
set of characters allowed in a Content-Transfer-Encoding mechanism name is
unspecified since without knowledge of whether specials or tspecials are
being used, the definition of a token is unclear.

A further conclusion is that a header can't contain both an addr-spec or
phrase syntax element and a MIME parameter syntax element without serious
parser ugliness.

Future header definitions should be very clear whether specials or
tspecials are used.

I note that 822bis is clear in this regard as it doesn't use implicit
free-insertion of linear whitespace between tokens.

I noticed this in the process of writing a strict RFC 822/MIME/DSN/MDN 
validation utility (I expect to make it public in a week or two on this
list).

		- Chris

Date: Tue, 30 Mar 1999 11:52:44 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 1:54 PM +0000 3/30/99, Charles Lindsey wrote:

>But wouldn't you then have to invent a new parameter, "Format=Superflowed"
>for messages conforming to the new format? And would it not be better to
>ensure that the present space-stuffing rule did not get in the way of this
>suggested extension (actually, I think it probably does not, but it should
>be thought about).
>
>Proletext, OTOH is a much bigger thing. I think it would have to be a
>whole new text type.

Any additional functionality would require a new value for the Format 
parameter, yes.  This would include your indent idea, proletext, or 
anything else.  I don't see that as such a big deal.  I'd rather get 
Format=Flowed out in its present form.  Any extra functionality is a 
potential obstacle, and would I think be better worked on after we 
get wider deployment and  operational experience with the current f=f.

Date: Tue, 30 Mar 1999 10:18:50 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Laurence Lundblade <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 11:40 AM +0000 3/18/99, Charles Lindsey wrote:
> In <v0410482eb31587e6dec9@[129.46.219.101]> Laurence Lundblade 
> <[email-masked]> writes:
>
>>I haven't looked at proletext, but will point out that any subtype of
>>text pretty much went down in flames when we tried sending some
>>samples around. We tried text/paragraph with a format that was
>>basically plain. Too many mailers default to treating this as an
>>attachment.
>
> Clearly such mailers are broken (i.e. non-compliant)

It's true, but...

> I think one had to be fairly brutal with non-compliant software. Just post
> messages obeying the rules, and when people with non-compliant software
> find themselves inconvenienced, then the pressure by them upon the writers
> of the software will produce results quicker than any other known method
> :-) .

This isn't really the old Internet any more. Most users don't know 
one format from another, what the standards, or who the IETF is. 
Compliance in their view is more based on whether it works with the 
existing installed base rather than any particular standard they 
don't even know exists.

Also, I do some amatuer statistics on mail I received and it boils 
down to most mail being sent by clients from one of the big five:
  Microsoft: MSMail, Outlook, WebTV, Hotmail
  Qualcomm: Eudora
  AOL/Netscape
  IBM: cc:mail, Notes
  U of Washington: Pine

I think you'll have to convince one or maybe two that proletext is a 
good thing to make it go forward. It seems also that if you think it 
has to be a separate type your going to run smack into HTML.

LL

To: [email-masked]
Xref: clerew local.mime:174
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Tue, 30 Mar 1999 13:54:24 GMT
Lines: 30
Sender: [email-masked]

In <[email-masked]> Randall Gellens <[email-masked]> writes:

>At 2:35 PM +0000 3/15/99, Charles Lindsey wrote:

>>What I am proposing merely requires a count of the SPACE characters at the
>>beginning of the line (with some allowance for TABs), to be counted AFTER
>>the '>' have been removed, but then treated in precisely the same way
>>(i.e. count them, remove them, wrap the line, and put them back).

>What you suggest does sound useful.  But I would still prefer to 
>advance Format=Flowed in its present, minimalist state, and see if it 
>gains acceptance.   If it does, and there is interest in doing more, 
>I'd be happy to pursue extensions such as your proposal or proletext 
>or something else.  (Note that your suggestion does conflict with the 
>space-stuffing rule.)

But wouldn't you then have to invent a new parameter, "Format=Superflowed"
for messages conforming to the new format? And would it not be better to
ensure that the present space-stuffing rule did not get in the way of this
suggested extension (actually, I think it probably does not, but it should
be thought about).

Proletext, OTOH is a much bigger thing. I think it would have to be a
whole new text type.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:173
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Date: Tue, 30 Mar 1999 13:49:48 GMT
Lines: 44
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> The problem with this discussion is that it has been hijacked by those who
>> want to discuss all sorts of bells and whistles that go beyond what I was
>> proposing. 

>net discussions have a way of doing this, especially when the participants
>don't first have agreement on what problem they're trying to solve.

Indeed. My immediate objective is clear - to provide a basic technology
for signing headers on top of which which various applications (e.g.
signing newgroup and cancel messages) can then be built.

>mail systems, and mail to news gateways, mung headers in so many ways
>that this may be asking too much.

Well I claim my draft makes it possible, but the details need to be
examined to see if it holds up.

>I think the real point is that signing only the headers is not very useful
>for email, for various reasons.  Trying to make it useful for email is
>probably a rathole.
>

Actually, if you sign headers (to prove they have not been significantly
altered en route, or that they originated where claimed) then you also
have to show that they are accompanied by the same body as they started
out with. I do that by having a Content-MD5 header and then including that
in the headers that are signed.

As to whether the feature is useful in mail in its own right, time will
tell. There might be applications where people wanted to send
authenticated datagrams using the mail transport mechanisms, but I leave
it to somebody else to invent that if they feel the urge. My real concern
for making it work in mail is that news articles sometimes find themselves
in the mail system (e.g. on the way to or from mailing lists).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Mon, 29 Mar 1999 18:53:48 -0800
To: [email-masked]
From: Randall Gellens <[email-masked]>
Subject: draft-gellens-format-04.txt
Cc: [email-masked], [email-masked], [email-masked], Chris Newman <[email-masked]>
Sender: [email-masked]

I've sent in draft-gellens-format-04.txt; it should be appearing soon 
in an I-D directory near you.  If you want to see it before then, it 
is available at 
<ftp://ftp.pensive.org/Public/Randy/draft-gellens-format-04.txt>.

Since the comments on -03 have generally been minor, I'm going to ask 
for an IETF last call on this version.  Please look it over and let 
me know of any problems.

--
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Ninety percent of the politicians give the other ten percent a bad name.
                                                      --Henry Kissinger

Date: Mon, 29 Mar 1999 17:38:13 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 2:35 PM +0000 3/15/99, Charles Lindsey wrote:

>I don't like the idea of an 'initial version' - it implies that a more
>complex version will follow later. Trouble is, people who implement the
>'initial version' will be none too happy if a 'revised version' follows
>too soon afterwards.

I think this is a good way for standards to proceed: Get consensus on 
an initial version that is a simple as possible and fixes the minimal 
subset of the problem.

If the standard succeeds and gets deployed, great.  If there is then 
interest in fixing an additional part of the problem, this can 
usually be done with minimal incremental cost, and with the benefit 
of the operational experience gained from deployment of the initial 
version.

>But in fact, what I suggest is not significantly more complex than what is
>proposed already (which is itself an 'uncomplex' as anything I have seen).
>
>It is already the case that the implementer has to keep a count of '>' at
>the beginning of a line, and then to do the line wrapping without those
>characters, and finally but them back after the wrapping.
>
>What I am proposing merely requires a count of the SPACE characters at the
>beginning of the line (with some allowance for TABs), to be counted AFTER
>the '>' have been removed, but then treated in precisely the same way
>(i.e. count them, remove them, wrap the line, and put them back).

What you suggest does sound useful.  But I would still prefer to 
advance Format=Flowed in its present, minimalist state, and see if it 
gains acceptance.   If it does, and there is interest in doing more, 
I'd be happy to pursue extensions such as your proposal or proletext 
or something else.  (Note that your suggestion does conflict with the 
space-stuffing rule.)

From: Keith Moore <[email-masked]>
To: [email-masked] (Charles Lindsey)
cc: [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
Date: Mon, 29 Mar 1999 11:56:45 -0500
Sender: [email-masked]

> >Then develop a solution that is tailored to those news applications,
> >but don't pretend that it is for use by end-users.  If you try to 
> >solve both problems at once you may end up making it too difficult.
> 
> The problem with this discussion is that it has been hijacked by those who
> want to discuss all sorts of bells and whistles that go beyond what I was
> proposing. 

net discussions have a way of doing this, especially when the participants
don't first have agreement on what problem they're trying to solve.

> I am only concerned with those applications where
> the integrity of the _headers_ needs to be ensured, and admittedly it is
> news that provides such applications presently. 

right.

> However, news articles
> tend to wander in and out of mail systems for a variety of reasons, which
> is why the system should preferably not break if it happens to pass
> through a mail route.

mail systems, and mail to news gateways, mung headers in so many ways
that this may be asking too much.

> >people who have done this before in the context of email, understand
> >why canonicalization is difficult.  the problems are similar enough that
> >people who haven't done this before should respect the experience of those
> >who have.  that doesn't inherently mean that new proposals cannot be 
> >examined, but neither can you demand that other people consider them.  
> >one of the things that experience teaches you is how to tell when you're 
> >getting near a rathole.
> 
> Yes, canonicalization is difficult, which is why I took a lot of trouble
> over it. So please look at what I proposed, and tell me where I got it
> wrong, rather than just saying "it is known to be impossible, so there is
> no point in trying to do it".

I think the real point is that signing only the headers is not very useful 
for email, for various reasons.  Trying to make it useful for email is 
probably a rathole.
 
Keith

To: [email-masked]
Xref: clerew local.mime:169
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Date: Mon, 29 Mar 1999 10:27:30 GMT
Lines: 73
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> >To be very blunt:
>> 
>> >(1) If I want to sign headers, I'll use multipart/signed of message/rfc822.
>> 
>> Fine. You can do that, but it is no solution for some news applications.

>Then develop a solution that is tailored to those news applications,
>but don't pretend that it is for use by end-users.  If you try to 
>solve both problems at once you may end up making it too difficult.

The problem with this discussion is that it has been hijacked by those who
want to discuss all sorts of bells and whistles that go beyond what I was
proposing. I have no desire to supersede the use of S/MIME and PGP/MIME
for those applications for which is adequate (if that is what you are
meaning by "end-users"). I am only concerned with those applications where
the integrity of the _headers_ needs to be ensured, and admittedly it is
news that provides such applications presently. However, news articles
tend to wander in and out of mail systems for a variety of reasons, which
is why the system should preferably not break if it happens to pass
through a mail route.

>> 
>> Agreed. My proposal says nothing about envelopes.

>strictly speaking, about half of the Usenet message header *is* an envelope.
>(i.e. it's intended for use by the news transport system)
>that's part of why the needs of Usenet are different than those of email.

Problem with mail is that envelope information sometimes duplicates (or
even contradicts) header information (e.g. in SMTP you can deliver a
message to A, even if all the To: fields, etc, say to deliver it to B). My
proposal is concerned only with the integrity of the headers that
accompany the message.

>> >(4) Any attempt at a canonicalization algorithm for mail header signing is
>> >    doomed to failure from the outset.  It will be ambiguous, too complex,
>> >    or inadequate.
>> 
>people who have done this before in the context of email, understand
>why canonicalization is difficult.  the problems are similar enough that
>people who haven't done this before should respect the experience of those
>who have.  that doesn't inherently mean that new proposals cannot be 
>examined, but neither can you demand that other people consider them.  
>one of the things that experience teaches you is how to tell when you're 
>getting near a rathole.

Yes, canonicalization is difficult, which is why I took a lot of trouble
over it. So please look at what I proposed, and tell me where I got it
wrong, rather than just saying "it is known to be impossible, so there is
no point in trying to do it".

>> 
>> >(B) If signed headers in email are attempted, I suspect the IESG will kill
>> >    the proposal so I don't have to expend energy fighting it.
>> 
>If the usenet-format people submit any kind of signature proposal to IESG,
>IESG is going to want (a) strong justification for why it should be approved
>and (b) absent very good reasons why this is a lot better than s/mime or 
>openpgp for general purpose uses, an applicability statement which says 
>what signed headers are good for and what they're not good for.

It's not supposed to be better than s/mime or openpgp, or to be for
general purpose uses. If people want to use it that way, then that is
their problem. It is to fill a gap that s/mime or openpgp do not fill (in
fact, it actually uses openpgp internally, as currently proposed).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

From: Keith Moore <[email-masked]>
To: Dan Kohn <[email-masked]>
cc: "'Brad Templeton'" <[email-masked]>, [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
Date: Fri, 26 Mar 1999 19:44:26 -0500
Sender: [email-masked]

> I'll just note in passing that the MTA could search its certificate
> revocation database for messages it forwards and attach appropriate
> revocations along with the message (perhaps as a header).

Relaying MTAs should NEVER modify messages, except to add postmarks.
Their job is to make sure the bits get from one end to the other intact.
The more that they modify messages, the less likely it is that what
the recipient gets even resembles what the sender sent.  There are
far too many MTAs that can't even get do simple relaying right
(how difficult is a copy loop anyway), without adding this kind of cruft.

If a user wants his message store wants to lookup CRLs, make
them available to his UA for dowload, etc. that's one thing, 
and I could see where a user might want his mail submission server
to sign outgoing mail on his behalf.  But MTAs in the middle 
have absolutely no business adding random attachments to messages.

Keith

From: Dan Kohn <[email-masked]>
To: "'Brad Templeton'" <[email-masked]>
Cc: [email-masked]
Subject: RE: Signed headers in email (was Re: Draft for signed headers)
Date: Fri, 26 Mar 1999 14:50:32 -0800
Sender: [email-masked]

I'll just note in passing that the MTA could search its certificate
revocation database for messages it forwards and attach appropriate
revocations along with the message (perhaps as a header).  The offline
MUA would then receive any associated certificate revocations at the
same time it receives the message, and would be able to deal with them
appropriately.  This sort of just-in-time certificate revocation
delivery seems to balance online and offline requirements appropriately.

		- dan

-----Original Message-----
From: Brad Templeton [mailto:[email-masked]]
Sent: Friday, 1999-03-26 12:58
To: Rick Troth
Cc: Keith Moore; Chris Newman; [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers)

On Fri, Mar 26, 1999 at 02:53:03PM -0600, Rick Troth wrote:
> > > MTAs, which operate on larger servers and which are also normally 
> > > online, can handle certificate revocation.
> > 
> > MTAs have no business making decisions on users' behalf.
> > It could be done at the message store, perhaps, but not in the MTA.
>  
> 	I second this.   MTAs should stay out of it,  leaving the 
> content alone,  simply passing traffic.   Message Store is an 
> implementation thing and could legitimately get involved. 

While I generally agree, I simply point out the problem that any
signature system with certificates needs to also support revocation of
certificates, and it is highly unlikely that individual mail clients,
especially when offline, will track the entire stream of revoked
certificates or have access to it.   Only larger servers, and online
servers, will have reasonable access to the revocation database.  This
could be MTAs or message delivery programs, as you say.   I simply wish
to point out that end-to-end verification of signatures is not really
workable with offline clients if you want to support signature
revocation.

The USENET-format group is deferring the whole question for now.

Date: Fri, 26 Mar 1999 12:57:46 -0800
From: Brad Templeton <[email-masked]>
To: Rick Troth <[email-masked]>
Cc: Keith Moore <[email-masked]>, Chris Newman <[email-masked]>, [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Fri, Mar 26, 1999 at 02:53:03PM -0600, Rick Troth wrote:
> > > MTAs, which operate on larger servers and which are also normally 
> > > online, can handle certificate revocation.
> > 
> > MTAs have no business making decisions on users' behalf.
> > It could be done at the message store, perhaps, but not in the MTA.
>  
> 	I second this.   MTAs should stay out of it,  leaving the 
> content alone,  simply passing traffic.   Message Store is an 
> implementation thing and could legitimately get involved. 

While I generally agree, I simply point out the problem that any
signature system with certificates needs to also support revocation of
certificates, and it is highly unlikely that individual mail clients,
especially when offline, will track the entire stream of revoked
certificates or have access to it.   Only larger servers, and online
servers, will have reasonable access to the revocation database.  This
could be MTAs or message delivery programs, as you say.   I simply wish
to point out that end-to-end verification of signatures is not really
workable with offline clients if you want to support signature revocation.

The USENET-format group is deferring the whole question for now.

Date: Fri, 26 Mar 1999 14:53:03 -0600 (CST)
From: Rick Troth <[email-masked]>
To: Keith Moore <[email-masked]>
cc: Brad Templeton <[email-masked]>, Chris Newman <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
Sender: [email-masked]

> > MTAs, which operate on larger servers and which are also normally 
> > online, can handle certificate revocation.
> 
> MTAs have no business making decisions on users' behalf.
> It could be done at the message store, perhaps, but not in the MTA.
 
	I second this.   MTAs should stay out of it,  leaving the 
content alone,  simply passing traffic.   Message Store is an 
implementation thing and could legitimately get involved. 
 
> Keith
 
-- 
Rick Troth at La Casita, Houston, Texas, USA 

From: Keith Moore <[email-masked]>
To: [email-masked], [email-masked]
cc: [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
Date: Fri, 26 Mar 1999 15:20:33 -0500
Sender: [email-masked]

> >To be very blunt:
> 
> >(1) If I want to sign headers, I'll use multipart/signed of message/rfc822.
> 
> Fine. You can do that, but it is no solution for some news applications.

Then develop a solution that is tailored to those news applications,
but don't pretend that it is for use by end-users.  If you try to 
solve both problems at once you may end up making it too difficult.

> >(3) If I want to sign SMTP envelope information, I'll use
> >    multipart/signed and RFC 2442 with a private agreement for the
> >    transport address/protocol to the recipient MTA.  The
> >    alternative hop-by-hop STARTTLS might provide less security, but is
> >    better than none and is deploying well.
> 
> Agreed. My proposal says nothing about envelopes.

strictly speaking, about half of the Usenet message header *is* an envelope.
(i.e. it's intended for use by the news transport system)
that's part of why the needs of Usenet are different than those of email.

(usenet's placement of envelope information in headers vs. email's
separation of headers and envelope, is the source of a lot of the 
problems associated with integrating usenet and email.  which is 
not to say that usenet did it wrong, but that the models are different
and to a large degree, incompatible.)

> >(4) Any attempt at a canonicalization algorithm for mail header signing is
> >    doomed to failure from the outset.  It will be ambiguous, too complex,
> >    or inadequate.
> 
> And that is a matter you decide by examining the proposals and finding
> holes in them. Not by asserting ex cathedra that no solution exists. I
> have proposed a canonicalization algorithm which may or may not work. It
> should be examined, discussed, and improved.

people who have done this before in the context of email, understand
why canonicalization is difficult.  the problems are similar enough that
people who haven't done this before should respect the experience of those
who have.  that doesn't inherently mean that new proposals cannot be 
examined, but neither can you demand that other people consider them.  
one of the things that experience teaches you is how to tell when you're 
getting near a rathole.

> 
> >(B) If signed headers in email are attempted, I suspect the IESG will kill
> >    the proposal so I don't have to expend energy fighting it.
> 
> If the usenet-format people submit a proposal to the IESG, I doubt IESG
> is going to kill it just because it _might_ get used in mail as well. 

If the usenet-format people submit any kind of signature proposal to IESG,
IESG is going to want (a) strong justification for why it should be approved
and (b) absent very good reasons why this is a lot better than s/mime or 
openpgp for general purpose uses, an applicability statement which says 
what signed headers are good for and what they're not good for.

Keith

To: [email-masked]
Xref: clerew local.mime:165 local.usenet:5934
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Fri, 26 Mar 1999 16:42:00 GMT
Lines: 28
Sender: [email-masked]

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>There are not two lists.  There are two ways to do things.   They are:

>	a) I list which headers I have signed (or decided not to include).
>	   Other programs can now add other headers.  We hope remote sites
>	   will realize they are not to be trusted.
>	b) I declare all headers MUST be signed, except ones I list.
>	   Other programs can only add headers on the exception list, or they
>	   must generate their own list (ie. they must understand signatures)

But the problem with method (b) is that injection agents (especially those
at big ISPs, or those in mail2news gateways) regard it as their absolute
right to add headers (Injected-By:, X-Trace:, NNTP-Posting-Host: and much
more), and to invent extra such headers whenever they feel like it. They
are not going to surrender that privilege easily.

Which is why (a) is much better. You can still easily tell which headers
were not covered by the signature. It is unlikely that they will include
anything that could seriously damage the integrity of the message
(especially if the poster uses the standard macro which carefully includes
all the critical ones).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:161 local.usenet:5931
Newsgroups: local.mime,local.usenet
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Reply-To: [email-masked], [email-masked]
Date: Fri, 26 Mar 1999 15:17:22 GMT
Lines: 67
Sender: [email-masked]

In <[email-masked]> [email-masked] writes:

>--==_Exmh_1716240888P
>Content-Type: text/plain; charset=us-ascii

>On Thu, 25 Mar 1999 11:30:44 PST, Brad Templeton said:
>> On Thu, Mar 25, 1999 at 12:51:19AM -0800, Chris Newman wrote:
>> > (D) If signed headers are done only in netnews, they will be removed by
>> >     the news->mail gateway, tunnelled through (1), or they likely won't
>> >     work in the gateway case at all.

No. Ideally, signed headers, originally made in news, will pass through
gateways of both kinds and still be verifiable. That is what my Draft set
out to make possible. Of course, there is bound to be some gateway
somewhere that munges to such an extent that it doesn't work, but it is
hoped that these will be in the minority

>> 
>> They probably won't be removed.  If they are kept the message can be 
>> re-gatewayed into news so long as it is intact. If it has been modified 
>> the re-gateway will fail or need to re-sign with its own key.

>OK.. I'll bite.  What does the fact that a piece of news is signed by
>the re-entry gateway tell me?  Somewhere near zero, since the gateway
>is in no position to verify that nothing untoward happened.  In fact,
>if the gateway is signing it, then we *know* the original signature
>is broken - if it was intact the gateway wouldn't need to sign it.

It depends on the what the gateway received. If it received a set of
signed headers and was able to verify the signature, but it intends now to
munge the headers before passing them on (to suit its own obscure site
policies), then it should state that it had verified the headers
(including exactly which ones) and sign its statement, and also sign the
complete thing again (well, there are various ways to do it, but see a
rather similar example in my Draft). Then everybody can see that the
originally signed article is still intact (assuming they believe what the
gateway has certified). There are enough hooks in my Draft to enable all
this to be done (whether gateways go to the trouble of using them all is
another matter, of course).

>So what this *really* says is "If it has been modified, the re-gateway
>will fail or need to be flagged as corrupted/modified".  A new signature
>isn't needed - only a header added ('X-mangled-in-transit: maybe?).  This
>seems to be the only header field that needs to be signed by the gateway,
>as nothing else is really trustable at that point.

If the thing did not verify correctly when it arrived at the gateway,
OTOH, then it would probably be better for the gateway to drop it. And
even if it passes it on, there is not much point in signing anything,
though it could assert, under its own signature, that the verification had
failed (yes, there is even a way to say that in my Draft).

But nobody should actually be removing any of these headers (unless the
article is dropped entirely, of course). If funny things have been
happening en route, then let all the evidence be preserved, so that people
can make up their own minds what to believe.

>-- 
>				Valdis Kletnieks
>				Computer Systems Senior Engineer
>				Virginia Tech

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:163
Newsgroups: local.usene,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Reply-To: [email-masked], [email-masked]
Date: Fri, 26 Mar 1999 16:29:44 GMT
Lines: 12
Sender: [email-masked]

In <[email-masked]> Ned Freed <[email-masked]> writes:

>I plan to review the proposal, if only to see what's being proposed, but
>I have a few hundred other things to do first.

Thank you. I look forward to hearing your comments in due course.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:162 local.usenet:5932
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Reply-To: [email-masked], [email-masked]
Date: Fri, 26 Mar 1999 16:27:49 GMT
Lines: 70
Sender: [email-masked]

In <[email-masked]> Chris Newman <[email-masked]> writes:

>To be very blunt:

>(1) If I want to sign headers, I'll use multipart/signed of message/rfc822.

Fine. You can do that, but it is no solution for some news applications.

>(2) MTAs are supposed to ignore headers anyway, so signed toplevel headers
>    have no value to mail transport systems.

Quite so. The signatures are for the benefit of the end-users.

>(3) If I want to sign SMTP envelope information, I'll use
>    multipart/signed and RFC 2442 with a private agreement for the
>    transport address/protocol to the recipient MTA.  The
>    alternative hop-by-hop STARTTLS might provide less security, but is
>    better than none and is deploying well.

Agreed. My proposal says nothing about envelopes.

>(4) Any attempt at a canonicalization algorithm for mail header signing is
>    doomed to failure from the outset.  It will be ambiguous, too complex,
>    or inadequate.

And that is a matter you decide by examining the proposals and finding
holes in them. Not by asserting ex cathedra that no solution exists. I
have proposed a canonicalization algorithm which may or may not work. It
should be examined, discussed, and improved.

>So:

>(A) I doubt signed headers will ever happen in email.

Time will tell.

>(B) If signed headers in email are attempted, I suspect the IESG will kill
>    the proposal so I don't have to expend energy fighting it.

If the usenet-format people submit a proposal to the IESG, I doubt IESG
is going to kill it just because it _might_ get used in mail as well. If
mail agents want to ignore it, that is their business. All I expect is
that they pass these strange headers through unmunged. If people in the
mail world subsequently find useful things to do with them, would that be
such a Bad Thing? News has found many things in mail that it can adapt
satisfactorily, but the migration of ideas between news and mail should be
a two-way traffic, do you not think?

>(C) If a form of signed headers other than (1) is deployed in email, it
>    probably won't work anyway so I don't have to worry about
>    implementing it.

Fine. So you are happy to let the news people develop this tool on their
own, and you are not going to complain if it turns out later that they
have omitted some small tweak that would have made it work much better in
mail? That is fine by me. But we shall in any case try to make it as
mail-proof as we can while we are about it.

>(D) If signed headers are done only in netnews, they will be removed by
>    the news->mail gateway, tunnelled through (1), or they likely won't
>    work in the gateway case at all.

Tunneling will certainly be used in many specific situations, but the
intention is to try and make it work with or without tunneling.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:164 local.usenet:5933
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Reply-To: [email-masked], [email-masked]
Date: Fri, 26 Mar 1999 16:35:14 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Keith Moore <[email-masked]> writes:

>> MTAs, which operate on larger servers and which are also normally 
>> online, can handle certificate revocation.

>MTAs have no business making decisions on users' behalf.
>It could be done at the message store, perhaps, but not in the MTA.

Absolutely so. If some bogus persons send me a message, I would still like
to know what is going on, even if I choose to disbelieve what is said.

But, OTOH, it could be a useful service for an MTA to spot that a signature
was bogus, and to inform me as it delivered the message. It could, for
example, do so by adding a header that says so (my draft even provides a
suitable header for the purpose, as it happens).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Thu, 25 Mar 1999 15:14:16 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 25, 1999 at 05:03:01PM -0500, [email-masked] wrote:
> OK.. I'll bite.  What does the fact that a piece of news is signed by
> the re-entry gateway tell me?  Somewhere near zero, since the gateway
> is in no position to verify that nothing untoward happened.  In fact,
> if the gateway is signing it, then we *know* the original signature
> is broken - if it was intact the gateway wouldn't need to sign it.

Indeed, it is unlikely that people would want to certify a re-entry
gateway.  However, if they wanted to certify one, they could, and it
could sign messages for re-insertion.   But I wouldn't advise it without
a lot of scrutiny.

Now some mail to news gateways will get certified, no doubt, but generally
their certificates will run only to the particular newsgroup they are
gatewaying into.   A general news to mail gateway that handles all groups
is unlikely.

Also possible is a news to mail gateway that works just for a site.  Since
it might well have (since it is operated by the site) a certificate letting
it sign messages as coming from that site, it could indeed handle any
newsgroup.  But it wouldn't handle messages from outside the site.

In general, going news -> mail -> news without preserving all news headers
and any news signature would be wrong, so it is not likely to get
certified as an activity.

To: Brad Templeton <[email-masked]>
Cc: [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
From: [email-masked]
Date: Thu, 25 Mar 1999 17:03:01 -0500
Sender: [email-masked]

--==_Exmh_1716240888P
Content-Type: text/plain; charset=us-ascii

On Thu, 25 Mar 1999 11:30:44 PST, Brad Templeton said:
> On Thu, Mar 25, 1999 at 12:51:19AM -0800, Chris Newman wrote:
> > (D) If signed headers are done only in netnews, they will be removed by
> >     the news->mail gateway, tunnelled through (1), or they likely won't
> >     work in the gateway case at all.
> 
> They probably won't be removed.  If they are kept the message can be
> re-gatewayed into news so long as it is intact.  If it has been modified
> the re-gateway will fail or need to re-sign with its own key.

OK.. I'll bite.  What does the fact that a piece of news is signed by
the re-entry gateway tell me?  Somewhere near zero, since the gateway
is in no position to verify that nothing untoward happened.  In fact,
if the gateway is signing it, then we *know* the original signature
is broken - if it was intact the gateway wouldn't need to sign it.

So what this *really* says is "If it has been modified, the re-gateway
will fail or need to be flagged as corrupted/modified".  A new signature
isn't needed - only a header added ('X-mangled-in-transit: maybe?).  This
seems to be the only header field that needs to be signed by the gateway,
as nothing else is really trustable at that point.

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_1716240888P
Content-Type: application/pgp-signature

--==_Exmh_1716240888P--

From: Keith Moore <[email-masked]>
To: Brad Templeton <[email-masked]>
cc: Chris Newman <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers) 
Date: Thu, 25 Mar 1999 16:52:18 -0500
Sender: [email-masked]

> MTAs, which operate on larger servers and which are also normally 
> online, can handle certificate revocation.

MTAs have no business making decisions on users' behalf.
It could be done at the message store, perhaps, but not in the MTA.

Keith

Date: Thu, 25 Mar 1999 12:05:18 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 25, 1999 at 11:19:21AM +0000, Charles Lindsey wrote:
> >	b) Mail has pretty fully moved to MIME support.
> 
> And we are pushing News in the same direction. But please note:
> 
> I AM NOT TRYING TO PROPOSE HEADER SIGNING AS THE NORMAL AUTHENTICATION
> METHOD FOR USENET. That is your interpretation, not supported by others. I
> am proposing it for specific applications (newgroups, cancels, etc, with
> more to follow when demand arises). I am quite content for it to coexist
> with existing MIME-based methods.

It would be pointless complexity to have multiple signing systems for
multiple types of messages.   We want to specify how to sign "articles" and
how to certify what articles do.   Control messages are just one type of
article.

> No. The requirements of mail and news are similar here. You do not want
> too many methods in either medium - you want standardised methods
> supported by many agents in both cases to promote the maximum
> interoperability. But OTOH "not too many" does NOT equate with "only one".

No, with mail, if I send you a message signed with S/MIME, all I need to know
is that *you* support S/MIME, no matter what the rest of the net does.

If I post a message to USENET, everybody on the net has to be able to handle
the method I use (if they want to be able to verify my signature.)  If you
have two methods to sign, then everybody on the net has to support them.

Date: Thu, 25 Mar 1999 11:30:44 -0800
From: Brad Templeton <[email-masked]>
To: Chris Newman <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 25, 1999 at 12:51:19AM -0800, Chris Newman wrote:
> (2) MTAs are supposed to ignore headers anyway, so signed toplevel headers
>     have no value to mail transport systems.

This creates a problem.  One of the most fundamental things people want
to verify at first is that a message comes from who it says it comes from.

This means verifying the From line and related lines at the very least.

However, mail user agents are not well suited to verify digital signatures
on mail from strangers.  They can handle people they know, but for strangers
they must rely on certificates signed by certification authorities.

But user agents, which are often offline when the process mail, have no
way to know if a certificate has been revoked.   It is not practical for
them to maintain a complete revocation database.   MTAs, which operate on
larger servers and which are also normally online, can handle
certificate revocation.

> (4) Any attempt at a canonicalization algorithm for mail header signing is
>     doomed to failure from the outset.  It will be ambiguous, too complex,
>     or inadequate.

While I agree it's risky, I think there are simple algorithms that can work.
The drafted algorithm is unlikley, as it attempts to understand the headers.

> (D) If signed headers are done only in netnews, they will be removed by
>     the news->mail gateway, tunnelled through (1), or they likely won't
>     work in the gateway case at all.

They probably won't be removed.  If they are kept the message can be
re-gatewayed into news so long as it is intact.  If it has been modified
the re-gateway will fail or need to re-sign with its own key.

To: [email-masked]
Xref: clerew local.mime:157 local.usenet:5906
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Thu, 25 Mar 1999 11:19:21 GMT
Lines: 69
Sender: [email-masked]

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>The reason they perhaps ignore it is they already have a couple of standards
>on the table, with working implementations in some cases.  These include
>S/MIME and multipart/signed, as well as a PGP form.   The chances of them
>taking another as a draft are slim to nil.

Sure, but those "couple of standards" are also useable in news (agreed the
S/MIME one is grossly unsuitable, but the PGP one is fine for news). So
themail position is not really different from ours. They could usefully
use an additional headers signing mechanism just like we could, except that
our need is far more pressing.

>But since there are a lot of big advantages to a signed header scheme (along
>with some disadvantages when it comes to mail) if USENET adopts one it is
>entirely possible that some mail tools would also adopt one, especially
>any merged tools.

Agreed. So it would be sensible that any scheme we devise should be
workable in mail as well. That limits us a bit, but not too much, as my
Draft was intended to show.

>USENET and mail have some needs in common, but a few big differences
>will make unification unlikley at present:
>	
>	a) Mail doesn't care nearly as much if the signatures are bulky
>	   and the certificates are bulkier.   Typical signed E-mail
>	   certs in S/MIME with X.509 can be multiple kilobytes -- the size
>	   of the average USENET post.

Sure, but how much is S/MIME actually used in email with those
certificates? For example, Lotus Notes is widely used, but doesn't use
multiple kilobytes (yes, I know it has a LEAK in it for the benefit of the
NSA, but that is by the way). But clearly, any header signature mechanism
would not be based on that. The headers added by pgpverify are just fine.

>	b) Mail has pretty fully moved to MIME support.

And we are pushing News in the same direction. But please note:

I AM NOT TRYING TO PROPOSE HEADER SIGNING AS THE NORMAL AUTHENTICATION
METHOD FOR USENET. That is your interpretation, not supported by others. I
am proposing it for specific applications (newgroups, cancels, etc, with
more to follow when demand arises). I am quite content for it to coexist
with existing MIME-based methods.

>	c) With mail  you know of and control who receives your mail.  Thus
>	   mail can happily support multiple competing methods, as long as
>	   the sender and recipients of each *specific* mail message
>	   understand the signature and certificate space.   With USENET,
>	   we have the "Highlander" problem -- "There can be only one."

No. The requirements of mail and news are similar here. You do not want
too many methods in either medium - you want standardised methods
supported by many agents in both cases to promote the maximum
interoperability. But OTOH "not too many" does NOT equate with "only one".

>	d) With mail, verification will be done by MTA or client (however
>	   MTA is safest) but there are no intermiediate points that want
>	   to verify, unlike news where all relayers SHOULD verify.  So there
>	   is less load problem and more choice.

I disagree totally.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Thu, 25 Mar 1999 01:15:32 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: Signed headers in email (was Re: Draft for signed headers)
To: Chris Newman <[email-masked]>
Cc: [email-masked], [email-masked]
Sender: [email-masked]

> Whatever happens, the issue isn't sufficiently compelling to merit wading
> through hundreds of long email messages.

Especially not when many if not most of the messages arrived during the 
week of the IETF meeting.

I plan to review the proposal, if only to see what's being proposed, but
I have a few hundred other things to do first.

				Ned

Date: Thu, 25 Mar 1999 00:58:33 -0800 (PST)
From: Chris Newman <[email-masked]>
Subject: Re: multipart + quoted-printable question
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: Keith Moore <[email-masked]>
Cc: [email-masked]
Sender: [email-masked]

On Thu, 11 Mar 1999, Keith Moore wrote:
> Now consider this body part (ending with a soft line break):
> 
> --foo
> Content-transfer-encoding: quoted-printable
> 
> =01=02=03=
> --foo
> 
> is the canonical form of this body part also (hex) 01 02 03?
> or is this just illegal because there's no following CRLF?

RFC 2045 pg 22:

    (3)   An "=" cannot be the ultimate or penultimate character
          in an encoded object.  This could be handled as in case
          (2) above.

		- Chris

Date: Thu, 25 Mar 1999 00:51:19 -0800 (PST)
From: Chris Newman <[email-masked]>
Subject: Signed headers in email (was Re: Draft for signed headers)
Originator-info: login-id=chris; server=THOR.INNOSOFT.COM
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

> > In <[email-masked]> Dave Barr <[email-masked]> writes:
> > I am disappointed that none of the IETF gurus (esp. those on
> > the ietf-822 list) have responded with the advice I asked for. Don't the
> > mail people care? Are they happy that the Usenet list alone should
> > prepare a draft that affects both news and mail?

To be very blunt:

(1) If I want to sign headers, I'll use multipart/signed of message/rfc822.
(2) MTAs are supposed to ignore headers anyway, so signed toplevel headers
    have no value to mail transport systems.
(3) If I want to sign SMTP envelope information, I'll use
    multipart/signed and RFC 2442 with a private agreement for the
    transport address/protocol to the recipient MTA.  The
    alternative hop-by-hop STARTTLS might provide less security, but is
    better than none and is deploying well.
(4) Any attempt at a canonicalization algorithm for mail header signing is
    doomed to failure from the outset.  It will be ambiguous, too complex,
    or inadequate.

So:

(A) I doubt signed headers will ever happen in email.
(B) If signed headers in email are attempted, I suspect the IESG will kill
    the proposal so I don't have to expend energy fighting it.
(C) If a form of signed headers other than (1) is deployed in email, it
    probably won't work anyway so I don't have to worry about
    implementing it.
(D) If signed headers are done only in netnews, they will be removed by
    the news->mail gateway, tunnelled through (1), or they likely won't
    work in the gateway case at all.

Whatever happens, the issue isn't sufficiently compelling to merit wading
through hundreds of long email messages.

		- Chris

Date: Wed, 24 Mar 1999 16:59:41 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Mon, Mar 22, 1999 at 10:51:22AM +0000, Charles Lindsey wrote:
> In <[email-masked]> Dave Barr <[email-masked]> writes:
> Quite so. Please can we STOP discussing the details, and get the politics
> right first. I am disappointed that none of the IETF gurus (esp. those on
> the ietf-822 list) have responded with the advice I asked for. Don't the
> mail people care? Are they happy that the Usenet list alone should
> prepare a draft that affects both news and mail?

The reason they perhaps ignore it is they already have a couple of standards
on the table, with working implementations in some cases.  These include
S/MIME and multipart/signed, as well as a PGP form.   The chances of them
taking another as a draft are slim to nil.

But since there are a lot of big advantages to a signed header scheme (along
with some disadvantages when it comes to mail) if USENET adopts one it is
entirely possible that some mail tools would also adopt one, especially
any merged tools.   And it would do an end run where it becomes popular.

USENET and mail have some needs in common, but a few big differences
will make unification unlikley at present:
	
	a) Mail doesn't care nearly as much if the signatures are bulky
	   and the certificates are bulkier.   Typical signed E-mail
	   certs in S/MIME with X.509 can be multiple kilobytes -- the size
	   of the average USENET post.  But they are only sent once or
	   a small number of times.  USENET posts, duplicated 100,000 times,
	   have a reason to avoid doubling.
	b) Mail has pretty fully moved to MIME support.  MIME messages are
	   very common, even though it isn't quite universal.  As such,
	   MIME based signing standards don't look ugly to mail users, nor
	   does their bulk cause much trouble.
	c) With mail  you know of and control who receives your mail.  Thus
	   mail can happily support multiple competing methods, as long as
	   the sender and recipients of each *specific* mail message
	   understand the signature and certificate space.   With USENET,
	   we have the "Highlander" problem -- "There can be only one."
	d) With mail, verification will be done by MTA or client (however
	   MTA is safest) but there are no intermiediate points that want
	   to verify, unlike news where all relayers SHOULD verify.  So there
	   is less load problem and more choice.

As such, while a USENET suitable system would work fine for mail, we're too
late.  Mail people have already designed systems that work in mail but won't
in USENET.  We must design our own system, and let mail people decide if they
would also like it.

Date: Wed, 24 Mar 1999 09:44:05 +0100
To: [email-masked] (Charles Lindsey), [email-masked]
From: Harald Alvestrand <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 18:40 18.03.99 +0000, Charles Lindsey wrote:
>In <v0410482eb31587e6dec9@[129.46.219.101]> Laurence Lundblade 
><[email-masked]> writes:
>
>>I haven't looked at proletext, but will point out that any subtype of 
>>text pretty much went down in flames when we tried sending some 
>>samples around. We tried text/paragraph with a format that was 
>>basically plain. Too many mailers default to treating this as an 
>>attachment.
>
>Clearly such mailers are broken (i.e. non-compliant)
>

So what else is new?
My favourite weirdness this week is that if Exchange *relays* a message
from SMTP to SMTP that contains a *charset* it does not understand, it turns
the body of that text/plain message into an attachment.
It seems 100% determined to create as many attachments as it can.

What fun....

                   Harald
-- 
Harald Tveit Alvestrand, Maxware, Norway
[email-masked]

To: [email-masked]
Xref: clerew local.mime:155 local.usenet:5886
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Mon, 22 Mar 1999 10:51:22 GMT
Lines: 84
Sender: [email-masked]

In <[email-masked]> Dave Barr <[email-masked]> writes:

>Time out folks.

>This thread is debating a draft which is not yet in the draft at hand.

>Let me refer to the article by Charles which started this thread.

>>The draft that I have prepared therefore tries to solve the problem in 
>>both environments. However, before we get too bogged down in the details 
>>of what I am proposing, I suggest we first establish the proper forum 
>>for this discussion, and the form that it should take. For this, I need 
>>some advice from the various IETF gurus on these lists.

Quite so. Please can we STOP discussing the details, and get the politics
right first. I am disappointed that none of the IETF gurus (esp. those on
the ietf-822 list) have responded with the advice I asked for. Don't the
mail people care? Are they happy that the Usenet list alone should
prepare a draft that affects both news and mail?

>>1.In the first place, should I aim it at a Draft Standard, or at an 
>>Experimental Protocol? Note that we shall eventually want to refer to it 
>>in the final USEFOR document, which is certainly supposed to be on the 
>>Standards Track, but it might still be useful (and quicker) to have an 
>>Experimental Protocol before then. 

>It seems to me that given the past, it would be a tough battle to
>get in in as a standards-track extension to our core draft (stage 2).
>Not only is there disagreement on the need for and architecture of
>any digital signature system, the format of the signature is also of hot
>debate.

Sure. This is not intended for the Usefor draft. It is an enabling
technology that should be established separately, perhaps even _before_
the main Usefor draft.

There was agreement earlier that Signed Headers were needed for various
applications (signing newgroups and unlocked cancels, verifying the
genuineness of moderated articles, and perhaps even having totally
verifiable hierarchies, though that is clearly further off). There may
also be purely mail applications (datagrams propagated by the mail
transport mechanisms intended to cause actions at remote sites).

We also have a Cancel-Lock draft already prepared, but we agreed not to
promulgate it until a signature mechanism was available to complement it.
My draft is a parallel effort to that Cancel-Lock draft.

My aim is to have a Signed-Headers document that can support various
news/mail applications (examples listed above). It is NOT intended to
define those applications itself - that is why I call it an "enabling
technology" - though it must be capable of supporting a wide range of
applications.

So please can we get the politics sorted out.

Is this an experimental protocol or standards track?

Is it to come out as a -usefor- draft, or an -ietf-822- draft, or a
-lindsey- draft, or do we need a separate IETF working group for it?

Should I establish a separate mailing list - or a list that simply creates
the union of usefor and ietf-822?

Do the mail people want in on the discussions (it is clear by the length
of this thread that the news people do, but that is to be expected since
they have the immediate need for the applications)?

>It's fun and exciting to debate neat new ideas, but our goal at present
>is to get our core standards document completed, and our next goal is
>to get any core extension drafts complete for inclusion in the final
>standards-track RFC.  Debating complex and controvertial issues
>is extremely time consuming, and we cannot afford the time now.

>If someone thinks we can get this in as a core extension, fine, finish
>the draft and we'll debate it when the next round comes.

>--Dave
>Your Chair.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Fri, 19 Mar 1999 19:22:03 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Fri, Mar 19, 1999 at 12:14:10PM +0000, Charles Lindsey wrote:
> 
> >Is this not correct?  Such a step is a non-necessary complexity.  If
> >the header list is tampered with, the signature will be immediately
> >invalidated anyway.
> 
> No it is essential (and it is in pgpverify too).
> 
> From: <good-guy>
> Message-Id: <fkew;lwfkew;kf>
> Signed: foobar From,Message-ID,Reply-To
>    rdskfrekerkgkrf;vker;
>    erkgvc,g[er5krlv;dllerpk';cd';lc';welv';r,';
> 
> Now a malicious interloper gets in the way and changes the message:
> 
> From: <good-guy>
> Message-Id: <fkew;lwfkew;kf>
> Reply-To: <bad-guy>
> Signed: foobar From,Message-ID
>    rdskfrekerkgkrf;vker;
>    erkgvc,g[er5krlv;dllerpk';cd';lc';welv';r,';
> 
> And if the first part of the Signed header was not covered in the
> signature, it will still verify.

That's the case if you use inclusive header lists.  However, the whole
reason people advocated inclusive header lists was that it would allow
later parties to add more headers to the article, so this consequence
is not totally unexpected.   What's salient about your example is that
the user explicitly listed reply-to on their list of headers signed but
did not include it.   Which is to say this is a hole that only occurs
through this deliberate action by the original poster.

In addition, the party reading the later article knows the Reply-to is not
signed, and they should not treat it as valid.

However, I agree that's not as good as we would like, it would be nice
if the article were entirely discarded and the end recipient did not have
to bother even noticing that the reply-to is in the untrusted portion of
the header.

(One reason I think the concept of an untrusted portion of a header is
asking for trouble.)

Still, I am loath to have anything but the simplest of canonicalized
hashing algorithms.  I think trying hash the header you are still trying
to generate is potentially as troublesome as the risk above but would ideally
want to fix both.   Though it's hard to find a way to do that, other than
to say "don't accept articles where any of the classic optional headers
are unsigned"  Ie. if Reply-to, Distribution, References, Summary etc. are
unsigned, reject it.

It might be a bit easier if you move to a tagged parameter Signed
header -- all new headers should use tagged parameters, not positionals.
That way you might be able to more easily do the exclusion of the
signature.  (In your proposal, excluding the sig  from the end, you stop
the header from being extensible)

To: [email-masked]
Xref: clerew local.mime:154 local.usenet:5844
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Fri, 19 Mar 1999 12:14:10 GMT
Lines: 35
Sender: [email-masked]

>My understanding is that your canonicalization algorithm, in order to
>prepare the hash stream, had the signer build the partial form of their
>own "signed" header, by canonicalizing the header list among other things
>until they had the form without the signature, and that this would be
>included in the hash stream.

>Is this not correct?  Such a step is a non-necessary complexity.  If
>the header list is tampered with, the signature will be immediately
>invalidated anyway.

No it is essential (and it is in pgpverify too).

From: <good-guy>
Message-Id: <fkew;lwfkew;kf>
Signed: foobar From,Message-ID,Reply-To
   rdskfrekerkgkrf;vker;
   erkgvc,g[er5krlv;dllerpk';cd';lc';welv';r,';

Now a malicious interloper gets in the way and changes the message:

From: <good-guy>
Message-Id: <fkew;lwfkew;kf>
Reply-To: <bad-guy>
Signed: foobar From,Message-ID
   rdskfrekerkgkrf;vker;
   erkgvc,g[er5krlv;dllerpk';cd';lc';welv';r,';

And if the first part of the Signed header was not covered in the
signature, it will still verify.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Fri, 19 Mar 1999 07:43:08 -0800
From: Brad Templeton <[email-masked]>
To: Brad Knowles <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Fri, Mar 19, 1999 at 12:33:09PM +0100, Brad Knowles wrote:
> On Thu, Mar 18, 1999, Brad Templeton <[email-masked]> wrote:
> 
>     The pgpverify scheme works now, and I submit that something similar
> would continue to work just fine.  You could extend it by allowing new
> keys to be distributed automatically via in-band communications channels,
> and if they're properly signed by the old keys they could automatically
> be added to the key ring.

If the only goal is to verify newgroup messages and the like, sure.

However, I presume we seek a means to do more than that.  To verify not
just these, but all control messages, and also to verify all postings in
moderated newsgroups, and eventually to offer the ability to verify any
posting, to prevent forgery.

If all we want is to verify newgroup, just about any scheme can work,
even one using PGP.

> 
>     I sincerely doubt that anyone is ever going to expect transit servers
> to validate each and every message.  I certainly don't want to run
> pgpverify (or something even heavier) on every single one of the 500k+
> messages our Diablo server processes on a daily basis.  I don't mind
> running something like pgpverify on control messages that
> create/modify/delete newsgroups, since those operations are relatively
> few and far between.

In a properly designed system that is exactly what I expect.  The CPU
load is entirely manageable today and will get lower with time as CPUs
grow faster.   Even if it weren't there are ways to deal with the problem.

It is important for transports to verify and not consumers because:
	a) You want to stop forgeries dead, as a favour to the sites that
	are not yet verifying messages.  Once major hubs verify, forgeries
	will have trouble getting very far.

	b) It saves a lot of complexity and time from clients.

	c) Because anybody who does verification also must track the
	   key revocation streams and maintain a key revocation database,
	   it's simply out of the question for things like dial-up clients
	   to do verification
	
	d) Because you want bad messages out of your overview, you want
	   them processed and the results in the overview database.

This does mean that the news server can do it and not the relay, but they
are very commonly the same.   And as noted there are reasons for relays to
do it.

This is no big whoop.  People are planning to have *routers* even verify
the security of packets in some apps.
> 
>     These machines are widely enough distributed and they should have
> enough extra CPU cycles to perform this function that we don't need to
> overload the central servers by asking them to look that intently at
> every single message.  It's enough that the newsreader does it and that
> the injecting agent checks it.

The newsreader can't do it.   It can't have access to the revocation lists,
and it doesn't help me to see an article in the menu, then download it and
have my reader say, "Sorry to waste your time, that was a forgery!"
> 
> 
>     Any news server along the path is certainly welcome to validate the
> posting if they so choose (and this fact should discourage attempts to
> forge signed postings), but they should not be required to validate each
> and every message that passes through them.

Nobody is required to validate, of course.  But if validation is done,
the servers are the place to do it.

Indeed, by having key hub sites validate, most other sites can goof off
about the matter.  Or use the digesting validation scheme I outlined.

Date: Fri, 19 Mar 1999 07:33:25 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Fri, Mar 19, 1999 at 11:22:51AM +0000, Charles Lindsey wrote:
> >It is wrong to expect there is only going to be one certificate space,
> >and that's what it would take to be 'gateway proof',
> 
> I expect no such thing. There may or may not be a single "certificate
> space" (I am not sure that the concept is even a helpful one) for a given
> application (such as Usenet Control messages). 

What I mean by that is this.   Every message does indeed rely on
some key at the top of its certificate chain.  The verifier needs to have
that key, presumably in advance based on normal USENET methods.

Since the goal is for every site on the net to be able to verify, then
everybody on the net must have that key.  That means the number of such
keys must be kept very small.   That set of keys is the certificate space,
the space of keys of top level CAs.
> 
> >The whole idea of certificates is you *don't* distribute them.  You don't
> >fetch them from servers.   USENET can't operate as I know it today
> >if you need to go off tor remote servers to process a message.
> 
> BULLSHIT! If someone invents a public key and never distributes it to
> anyone else see it, what use is it?

The whole point of certificates is you don't distribute the keys of regular
users.  You distribute only the very small number of keys of the highest
level CAs.

For this to work as a USENET verifyer, you can't be having to fetch keys
from central servers on a regular basis.   I expect there to be less than
100 of these "keeper" keys as I call them -- keys of the top level CAs
and similar parties.  As such, you might manually install them, or might
arrange, if you are on the internet, to fetch one in the very rare event
you see one you don't know.

> How much certificate material is contained in each message, and how much
> is obtained from well-known servers (and doubtless cached locally) will
> differ according to the application. In the Usenet case, I imagine most of
> them would be distributed via newsgroups established for the purpose.

I would imagine the top level keys would be distributed that way, or as
control messages, and be available in other forms.
> 
> >The problem is that you don't want people posting messages you can't
> >verify, and if you post a message, you want to be sure everybody can
> >verify it.
> 
> No. People who regard it as important will want to verify it.

That's not securing the net, which may be the difference of opinion here.

Securing the net only works, in the end, if you can reject messages
that are not signed.   Not just confirm messages that are signed.  That
means that, within your "space", *all* messages are signed.

That space might be "all newgroups" or "all posts to moderated group X"
or "all control messages" or whatever.

And down the road it eventually means "all messages in groups or subnets
using authentication".
> 
> Why are you so concerned to stop people from doing things that you,
> personally, don't like? Even supposing there were such a thing as "the
> E-mail certificate world" (actually there will probably be several of them
> :-( ) why should people on Usnet veryify them if them happen to turn up?

One major goal is preventing forgery.  That means not letting people post
as me.   To do that, I need a certificate saying I am certified to own
my E-mail address.   That will be the usual certificate of the E-mail
world.   They also are useful on USENET.

But to stop forgery, it is necessary not just that I have a certificate
for my E-mail address (and that nobody else have one).  It is also necessary
that sites discard any article that claims to come from me and doesn't
have a certificate.  And in fact, it is necessary that sites discard any
article that claims to come from *anybody* and isn't signed with a
certificate.  Because otherwise people can still post from addresses
like "[email-masked]" and still cause replies to come to me,
or from "[email-masked] (Brad Templeton)" and still have most people
off the cuff think it was me.

Now you may not have included stopping forgery among the goals for
signing messages, but I do.

> 
> >And right now the rest of the certificate world is gravitating to x.509 which
> >is really not suitable for USENET.
> 
> I think X.509 is dying.

I wish.  I went to the RSA security conference last month and it was all the
rage.

Date: Fri, 19 Mar 1999 12:33:09 +0100
To: Brad Templeton <[email-masked]>, [email-masked], [email-masked]
From: Brad Knowles <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Thu, Mar 18, 1999, Brad Templeton <[email-masked]> wrote:

>It is wrong to expect there is only going to be one certificate space,
>and that's what it would take to be 'gateway proof', unless you refer
>to gateways that move out of one certificate space into another and back
>to the original.

    The other solution is to make gateways ignorant of certificates and
certificate spaces (they just see it as more article body content that
they don't do anything with), and to ensure that when the certificate
spaces are defined that they be as transparent as possible to any known
current or likely future gateway systems.

>The whole idea of certificates is you *don't* distribute them.  You don't
>fetch them from servers.   USENET can't operate as I know it today
>if you need to go off tor remote servers to process a message.

    The pgpverify scheme works now, and I submit that something similar
would continue to work just fine.  You could extend it by allowing new
keys to be distributed automatically via in-band communications channels,
and if they're properly signed by the old keys they could automatically
be added to the key ring.

    You could also extend it by referencing a central key server (or have
it made available via standardized methods on your favourite web server),
and if the validating agent wishes to validate a message that has a key
it does not have locally, it could go retrieve the key interactively. 
For those systems that don't have direct access to the network for
retrieving the keys, there would need to be a mechanism whereby they
could communicate with gateways that could perform this function for them
and then forward the key via in-band or out-of-band communications methods.

    You could also use the certificate system you describe.

    The entire point of allowing for unspecified key distribution methods
and certificate spaces is that all of the above work better or worse for
certain applications, and that no one solution will necessarily be the
best in all cases.  Therefore we allow for multiple ways to solve the
same problem, depending on the specific needs of the particular
application, and it is not within our charter for this group and this
document to formally and explicitly enumerate all the possibilities
within this particular area.

>The problem is that you don't want people posting messages you can't
>verify, and if you post a message, you want to be sure everybody can
>verify it.

    I sincerely doubt that anyone is ever going to expect transit servers
to validate each and every message.  I certainly don't want to run
pgpverify (or something even heavier) on every single one of the 500k+
messages our Diablo server processes on a daily basis.  I don't mind
running something like pgpverify on control messages that
create/modify/delete newsgroups, since those operations are relatively
few and far between.

    IMO, both the signing and validation processes are best left to the
end nodes -- the computer that one uses to compose and transmit an
article, and the computer that one uses to retrieve and read an article.

    These machines are widely enough distributed and they should have
enough extra CPU cycles to perform this function that we don't need to
overload the central servers by asking them to look that intently at
every single message.  It's enough that the newsreader does it and that
the injecting agent checks it.

    Any news server along the path is certainly welcome to validate the
posting if they so choose (and this fact should discourage attempts to
forge signed postings), but they should not be required to validate each
and every message that passes through them.

>            If you let people post a message using keys and certificates
>from the E-mail certificate world, you are in effect saying all USENET
>sites have to understand E-mail certificates.   Or those of any other world
>we will allow in.

    The e-mail model of MTAs that transmit signed articles without
validating them is one I think that would be good to emulate, at least to
this degree.  I'm not entirely sure that we would want the one true way
of signing Usenet postings to be the same heavy and verbose X.509v3
protocol that they seem to be leaning towards, however.

    I complain that one of the biggest problems on Usenet today is the
volume, and x.509v3 would certainly be a major contributor to the volume
problem.

-- 
  These are my opinions -- not to be taken as official Skynet policy
 ____________________________________________________________________
|o| Brad Knowles, <[email-masked]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News/mail/FTP Admin   Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o| http://www.skynet.be                     Belgium               |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
   Usenet is not the web.  Just because the web handles some things
 poorly is not a good reason to apply those same solutions to Usenet.

To: [email-masked]
Xref: clerew local.mime:153 local.usenet:5843
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Fri, 19 Mar 1999 12:08:13 GMT
Lines: 36
Sender: [email-masked]

In <[email-masked]> Brad Knowles <[email-masked]> writes:

>On Wed, Mar 17, 1999, Charles Lindsey <[email-masked]> wrote:

>>That doesn't work for News, unfortunately, where this feature is
>>considered to be essential. Encapsulation doesn't solve the problem.

>    Why not?  Anyone who has their public key can validate the signed
>headers, and if anything doesn't match that should (e.g., "From:",
>"Date:", etc...) the message can either be dropped on the floor or the
>unmatching unsigned headers can be replaced by the corresponding signed
>headers.

>    Maybe I'm missing something fundamental here, but I don't see why
>encapsulation won't work.

Because Netnews Control headers MUST be in the real headers to work. That
is the point that the man who raised this problem had not grasped.

Now there are possible horrid schemes involving duplicating the message
headers in some multipart, but that is an undesirable solution to the
problem if we can agree on a better way. I believe my draft shows that a
better way is possible (it is just the canonicalization algorithm that
needs to be got right).

BTW, I would welcome some input from the Mail Gurus on the ietf-822 list
as to how this matter should be progressed through the IETF processes (I
asked for such feedback in my original posting). I don't think it is a
good idea that just the USEFOR people should be discussing a draft that is
applicable to both media.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:152 local.usenet:5840
Newsgroups: local.mime,local.usenet
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Fri, 19 Mar 1999 11:22:51 GMT
Lines: 69
Sender: [email-masked]

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>On Thu, Mar 18, 1999 at 01:37:54PM +0000, Charles Lindsey wrote:
>> But even then, we need to make it gateway proof if at all possible.

>I don't see how to do this, for reasons including the ones I cited.

Why not read my draft, which explains how to do it.

>It is wrong to expect there is only going to be one certificate space,
>and that's what it would take to be 'gateway proof',

I expect no such thing. There may or may not be a single "certificate
space" (I am not sure that the concept is even a helpful one) for a given
application (such as Usenet Control messages). 

>> No. My Draft carefully left it up to each 'application' as to how
>> certificates were distributed.

>I don't see how that could work in any efficient manner.  Can you describe
>how?

It may be that I need an additional 'application' field in the Signed header.

>The whole idea of certificates is you *don't* distribute them.  You don't
>fetch them from servers.   USENET can't operate as I know it today
>if you need to go off tor remote servers to process a message.

BULLSHIT! If someone invents a public key and never distributes it to
anyone else see it, what use is it?

>The whole reason to use certificates is to avoid that.   So that every
>message, contains, within itself, the means to verify it using only one
>of a small set of well known keys that everybody will have on hand or can
>get immediately.

So, you admit that some keys must have been distributed somehow.

How much certificate material is contained in each message, and how much
is obtained from well-known servers (and doubtless cached locally) will
differ according to the application. In the Usenet case, I imagine most of
them would be distributed via newsgroups established for the purpose.

>The problem is that you don't want people posting messages you can't
>verify, and if you post a message, you want to be sure everybody can
>verify it.

No. People who regard it as important will want to verify it.

>  If you let people post a message using keys and certificates
>from the E-mail certificate world, you are in effect saying all USENET
>sites have to understand E-mail certificates.

Why are you so concerned to stop people from doing things that you,
personally, don't like? Even supposing there were such a thing as "the
E-mail certificate world" (actually there will probably be several of them
:-( ) why should people on Usnet veryify them if them happen to turn up?

>And right now the rest of the certificate world is gravitating to x.509 which
>is really not suitable for USENET.

I think X.509 is dying.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Thu, 18 Mar 1999 22:44:11 -0800
From: Brad Templeton <[email-masked]>
To: Russ Allbery <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 18, 1999 at 02:57:33PM -0800, Russ Allbery wrote:
> Charles Lindsey <[email-masked]> writes:
> > Brad Templeton <[email-masked]> writes:
> >> a) Used PGP, a then-proprietary external program
> 
> > But which nobody had any difficulty using
> 
> Um, pardon?  That's rather far from true.  There are several news sites
> that have simply turned off control message processing altogether because
> they're unwilling to pay a couple grand to NAI for a commercial PGP
> license.  PGP also has one of the most arcane and idiotic interfaces that

Actually, the theory is that openpgp clears some of this up but it
really doesn't make it a wise choice for USENET.

OpenPGP is a generalized system for encoding signatures and limited
certificates using a variety of algorithms in a packed-bits format.
It's certificate system, as far as I understand it, both has no particular
functions to certify USENET attributes, and is based on the web of trust
concept, which while interesting for some attributes, does not seem likely
to work for USENET signing and certifying.

To store signatures no encoding method of significance is needed.  A
signature is just a number.  You encode it in base64.  It needs no other
fields or attributes.  (Though just to be safe, the signed header, like
all headers, should be in MIME extensible header style with fieldname
tagged attributes.)

A likely choice for signature is DSA, which means the signature is a
320 bit number (two 160 bit numbers) to be encoded in 53 or 54 base6
characters, depending on if you view it as 2 numbers or one.

That's about it for the signature.  All the real complexity is in
the certificates.  Certificates are simply a short digitally signed
statement that says, "I certify that key X has attributes Y", signed
with the certifier's key.   The attributes are:
	a) Key-related attributes like key name
	b) Certificate attributes like expiration date and trust level
	c) USENET attributes like "Can use E-mail address foo" or
	   "Can newgroup in comp.*" or "Can cancel message-ids that
	   match <[email-masked]>" and so on.   

I have a list of the things we might want to certify on USENET on my web
site at www.templetons.com/usenet-format

These needs don't really tie in well with openPGP.

The certificates need a language of some sort to express the attributes.
I had designed one, but am not particularly wedded to it. Any language
that says the things we need to say in a compact, extensible, easy to
handle fashion is fine by me.   SPKI is one candidate others are working
on, though it has a few minor problems we may want to look into from my
basic examination.

To: [email-masked]
Cc: [email-masked]
Subject: Re: Draft for signed headers
From: Russ Allbery <[email-masked]>
Date: 18 Mar 1999 14:57:33 -0800
Lines: 35
Sender: [email-masked]

Charles Lindsey <[email-masked]> writes:
> Brad Templeton <[email-masked]> writes:

>> Wow, it's really grown if that's true, which is good to see.  And
>> indicates that support for a 3rd party article signing standard would
>> be strong right away.  PGPverify wasn't too bad as a first stab, though
>> it's main flaws were:

>> a) Used PGP, a then-proprietary external program

> But which nobody had any difficulty using

Um, pardon?  That's rather far from true.  There are several news sites
that have simply turned off control message processing altogether because
they're unwilling to pay a couple grand to NAI for a commercial PGP
license.  PGP also has one of the most arcane and idiotic interfaces that
I've had the misfortune of dealing with, and there was lots of pain
involved in finding some sort of portable canonicalization that deals with
incompatible ways of handling whitespace across different versions.

GnuPG is *much* nicer, but not yet viable for systems without a
/dev/random (although it's getting close).  NAI's PGP is not a viable
technology to base an authentication infrastructure on.

Not to mention....

windlord:~> host letters.mcafee.com
letters.mcafee.com      A       206.156.198.120
windlord:~> host -t txt 120.198.156.206.rbl.maps.vix.com
120.198.156.206.rbl.maps.vix.com        TXT     "Blackholed - see <URL:http://maps.vix.com/cgi-bin/lookup?206.156.198.120>"

...the fact that they're not particularly good Internet citizens.

-- 
Russ Allbery ([email-masked])         <URL:http://www.eyrie.org/~eagle/>

Date: Thu, 18 Mar 1999 10:27:29 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 18, 1999 at 01:24:33PM +0000, Charles Lindsey wrote:
> In <[email-masked]> Brad Templeton <[email-masked]> writes:
> 
> >On Wed, Mar 17, 1999 at 11:54:40AM -0500, [email-masked] wrote:
> >> > 
> >> > There is no need to "sign" a "signed" header.   Can you tell me why you
> >> > would want to do this?   A "signed" header (and any certificate) is
> >> > verifiable on its own when paired with the signed headers and body.  I
> >> 
> 
> >No, the standard proposed a complex mechanism of self-signing parts of
> >the signed header (a canonicalized expansion of the header list, and
> >all the other parts except the signature)
> 
> There is an example in my Draft of a Signed header that is itself signed.
> I think it was a realistic case, though I agree it will not be common.
> 

My understanding is that your canonicalization algorithm, in order to
prepare the hash stream, had the signer build the partial form of their
own "signed" header, by canonicalizing the header list among other things
until they had the form without the signature, and that this would be
included in the hash stream.

Is this not correct?  Such a step is a non-necessary complexity.  If
the header list is tampered with, the signature will be immediately
invalidated anyway.

Date: Thu, 18 Mar 1999 11:50:56 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Thu, Mar 18, 1999 at 01:37:54PM +0000, Charles Lindsey wrote:
> If it is laudable, then we should make all reasonable efforts to achieve
> it. It the mail gurus don't want to play, then the Usenet people can go it
> alone. But even then, we need to make it gateway proof if at all possible.

I don't see how to do this, for reasons including the ones I cited.

It is wrong to expect there is only going to be one certificate space,
and that's what it would take to be 'gateway proof', unless you refer
to gateways that move out of one certificate space into another and back
to the original.

Frankly, such gateways have always been deprecated for USENET and I see
no reason to start having them.  However, any USENET signing scheme, if
simply preserved in the headers, will allow such a re-gateway with safety,
which is better than what we had before!

> No. My Draft carefully left it up to each 'application' as to how
> certificates were distributed. Insofar as signing Usenet Control messages
> is an application (the main/only one on the table so far) then we may well
> define an infrastructure for that purpose. In the absence of specific
> applications, people will presumably have to get certificates from the
> usual PGP servers and decide for themselves how much of the web-of-trust
> to believe.

I don't see how that could work in any efficient manner.  Can you describe
how?

The whole idea of certificates is you *don't* distribute them.  You don't
fetch them from servers.   USENET can't operate as I know it today
if you need to go off tor remote servers to process a message.

The whole reason to use certificates is to avoid that.   So that every
message, contains, within itself, the means to verify it using only one
of a small set of well known keys that everybody will have on hand or can
get immediately.

You have to distribute those keys, but they are such a small set that it
can even be done manually if need be (though it's nice to work out automated
systems.)

> 
> That is a messy solution to be avoided if possible. Nevertheless, my draft
> makes provision for that where necessary.

The problem is that you don't want people posting messages you can't
verify, and if you post a message, you want to be sure everybody can
verify it.  If you let people post a message using keys and certificates
from the E-mail certificate world, you are in effect saying all USENET
sites have to understand E-mail certificates.   Or those of any other world
we will allow in.

And right now the rest of the certificate world is gravitating to x.509 which
is really not suitable for USENET.

To: [email-masked]
Xref: clerew local.mime:148 local.usenet:5822
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Thu, 18 Mar 1999 13:37:54 GMT
Lines: 66
Sender: [email-masked]

PLEASE CROSS POST THESE DISCUSSIONS TO BOTH LISTS.

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>On Wed, Mar 17, 1999 at 03:49:28PM +0000, Charles Lindsey wrote:

>> In <[email-masked]> Jean-Francois Stenuit <[email-masked]> writes:
>> 
>> 
>> The reasons may be different, but the mechanisms can be the same.
>> 
>> The chief reason for having the same mechanisms is that articles may get
>> gatewayed from news to mail, or vice versa, and one wishes the signature
>> mechanisms to survive the transition so far as may be possible.
>I don't think that's an attainable goal, laudable as it may be, and as
>such I will advise against major effort being spent on it.

If it is laudable, then we should make all reasonable efforts to achieve
it. It the mail gurus don't want to play, then the Usenet people can go it
alone. But even then, we need to make it gateway proof if at all possible.

>E-mail groups are already way ahead of us in working out E-mail signature
>schemes.  All the competing schemes, however, are not really workable
>inside USENET.

>In addition, any signature scheme will have a "certificate space" where
>certifying keys are expected to be known and respected.  It's far from
>sure that we can overlap the spaces of USENET and E-mail in this area.

No. My Draft carefully left it up to each 'application' as to how
certificates were distributed. Insofar as signing Usenet Control messages
is an application (the main/only one on the table so far) then we may well
define an infrastructure for that purpose. In the absence of specific
applications, people will presumably have to get certificates from the
usual PGP servers and decide for themselves how much of the web-of-trust
to believe.

>While it's true that the most common USENET certificate will simply
>say "This keyholder is entitled to act as E-mail address [email-masked]" and
>that is in common with an E-mail certificate's goals, the truth is all
>the E-mail certificates are *huge*, far too large to be used in USENET.
>(many exceed the average length of a USENET message!)

>Howevever, that doesn't meant they can't interoperate.  The way they
>would is either a gateway that re-signs a message and says, "I checked
>the sig from the other system, and I now sign using our system that it
>was valid.  If you trust me, and you trust the other system, you can trust
>this message."

That is a messy solution to be avoided if possible. Nevertheless, my draft
makes provision for that where necessary.

>Or more commonly, one can get one's certificate mapped.  So if you have
>an E-mail certificate certifying your E-mail address, you don't use it in
>your USENET postings, instead you ask a CA to convert it into a USENET
>E-mail address certificate, and you use that.

>If E-mail systems had a system under proposal with a decently sized
>certificate that met USENET needs, that would be great, but it ain't
>happening.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:146 local.usenet:5820
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Thu, 18 Mar 1999 13:24:33 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>On Wed, Mar 17, 1999 at 11:54:40AM -0500, [email-masked] wrote:
>> > 
>> > There is no need to "sign" a "signed" header.   Can you tell me why you
>> > would want to do this?   A "signed" header (and any certificate) is
>> > verifiable on its own when paired with the signed headers and body.  I
>> 

>No, the standard proposed a complex mechanism of self-signing parts of
>the signed header (a canonicalized expansion of the header list, and
>all the other parts except the signature)

There is an example in my Draft of a Signed header that is itself signed.
I think it was a realistic case, though I agree it will not be common.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:147 local.usenet:5821
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Date: Thu, 18 Mar 1999 13:27:42 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> Brad Templeton <[email-masked]> writes:

>Wow, it's really grown if that's true, which is good to see.  And indicates
>that support for a 3rd party article signing standard would be strong
>right away.   PGPverify wasn't too bad as a first stab, though it's main
>flaws were:
>	a) Used PGP, a then-proprietary external program
But which nobody had any difficulty using
>	b) Needed a bulky header list
No bulkier than other likely mechanisms
>	c) Didn't sign the body
Yes it did.

In fact, its most frequent reason for failure was the appearance or
disappearance of trailing empty lines in the body.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Thu, 18 Mar 1999 10:49:07 -0500 (EST)
From: Christopher Hicks <[email-masked]>
To: Charles Lindsey <[email-masked]>
cc: [email-masked]
Subject: Re: draft-gellens-format-03.txt
Organization: Flamingo Internet Navigators
Sender: [email-masked]

On Thu, 18 Mar 1999, Charles Lindsey wrote:

> I think one had to be fairly brutal with non-compliant software. Just
> post messages obeying the rules, and when people with non-compliant
> software find themselves inconvenienced, then the pressure by them
> upon the writers of the software will produce results quicker than any
> other known method :-) .

Many people won't want to send mail that can't be read by Outlook or AOL.  
For the most part mailing lists are pure text.  But you still see people
sending Outlook HTML/text MIME concoctions.  I'm sure some have
.procmail'd Outlook email because it can so annoying.

I'd like a tool that sends mail to postmaster for any domain that sends me
noncompliant email.  How many date formats will pine have to support when
Date: was defined way back in RFC822?  My favorite is the spam tools that
send dates with slashes: 1/22/99!  Maybe the spammer tools are not y2k
compliant.  :-)

Anyway, pressure and inconvenience aren't very reliable in light of users
believing that Microsoft (or IBM) has all the answers.  Brutality will
affect the user of 'our' software too and you have to be the majority for
such brutality to be effective.  Microsoft is the majority despite their
utter lack of RFC compliance.  Which leaves you with gorilla warfare.

-- 
</chris>

Neither Sweat, nor blood, nor frustration, or lousy manuals
nor missing parts, or wrong parts shall keep me from my task.

UNIX *is* user-friendly.  It's just picky about who its friends are.

Date: Thu, 18 Mar 1999 12:53:48 +0100
To: Brad Templeton <[email-masked]>, [email-masked]
CC: [email-masked], [email-masked], [email-masked]
From: Brad Knowles <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Wed, Mar 17, 1999, Brad Templeton <[email-masked]> wrote:

>The question is, what percentage of news sites are confirming newgroup
>with pgpverify?  I haven't tracked in a while, but back when I was
>using it (1.5 years ago) the answer was few.

    My impression is that the percentage has risen pretty dramatically. 
I believe that DNEWS does this by default (I'm not sure about Diablo),
and enough sites out there have been hit by excessive bogus newsgroup
creation that many servers get configured for this first thing right
out-of-the box.

-- 
  These are my opinions -- not to be taken as official Skynet policy
 ____________________________________________________________________
|o| Brad Knowles, <[email-masked]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News/mail/FTP Admin   Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o| http://www.skynet.be                     Belgium               |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
   Usenet is not the web.  Just because the web handles some things
 poorly is not a good reason to apply those same solutions to Usenet.

To: [email-masked]
Xref: clerew local.mime:145
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Thu, 18 Mar 1999 11:40:30 GMT
Lines: 21
Sender: [email-masked]

In <v0410482eb31587e6dec9@[129.46.219.101]> Laurence Lundblade <[email-masked]> writes:

>I haven't looked at proletext, but will point out that any subtype of 
>text pretty much went down in flames when we tried sending some 
>samples around. We tried text/paragraph with a format that was 
>basically plain. Too many mailers default to treating this as an 
>attachment.

Clearly such mailers are broken (i.e. non-compliant)

I think one had to be fairly brutal with non-compliant software. Just post
messages obeying the rules, and when people with non-compliant software
find themselves inconvenienced, then the pressure by them upon the writers
of the software will produce results quicker than any other known method
:-) .

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:142 local.usenet:5764
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Date: Wed, 17 Mar 1999 15:49:28 GMT
Lines: 24
Sender: [email-masked]

In <[email-masked]> Jean-Francois Stenuit <[email-masked]> writes:

>On Tue, 16 Mar 1999, Charles Lindsey wrote:

>> There is a strongly perceived need within the Usenet-Format group for
>> a reliable method for the digital signing of headers. It would, on the
>> face of it, be foolish to devise a mechanism applicable to news that
>> would not work also for email, even though that does indeed make the
>> mechanism more complex.

>IMHO, there should be two different schemes for NNTP and SMTP messages,
>because the reason for signing the header is different.

The reasons may be different, but the mechanisms can be the same.

The chief reason for having the same mechanisms is that articles may get
gatewayed from news to mail, or vice versa, and one wishes the signature
mechanisms to survive the transition so far as may be possible.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:143 local.usenet:5765
Newsgroups: local.usenet,local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: Draft for signed headers
Date: Wed, 17 Mar 1999 15:55:14 GMT
Lines: 25
Sender: [email-masked]

In <[email-masked]> Paul Hoffman / IMC <[email-masked]> writes:

>I don't see a need for this in the email world. In our many discussions of 
>the topic, two themes have always emerged:
>- Signing in nice but is rarely needed
>- When it is needed, you can encapsulate the message with its headers and 
>sign the result

>--Paul Hoffman, Director
>--Internet Mail Consortium

That doesn't work for News, unfortunately, where this feature is
considered to be essential. Encapsulation doesn't solve the problem.

Clearly, it would be possible for News to adopt some standard for this
that was entirely imcompatible with email, but that would cause problems
with news-mail gateways (in either direction) and, moreover, deliberate
incompatibility does not seem like a Good Thing unless it is technically
unavoidable.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Wed, 17 Mar 1999 09:06:07 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Wed, Mar 17, 1999 at 06:02:55PM +0100, Andreas Barth wrote:
> On Wed, 17 Mar 1999, Brad Templeton wrote:
> > The question is, what percentage of news sites are confirming newgroup
> > with pgpverify?  I haven't tracked in a while, but back when I was
> > using it (1.5 years ago) the answer was few.
> I haven't tracked it worldwide, but here in Munich it's about 80% of the 
> "important" news sites.

Wow, it's really grown if that's true, which is good to see.  And indicates
that support for a 3rd party article signing standard would be strong
right away.   PGPverify wasn't too bad as a first stab, though it's main
flaws were:
	a) Used PGP, a then-proprietary external program
	b) Needed a bulky header list
	c) Didn't sign the body

Date: Wed, 17 Mar 1999 18:02:55 +0100 (MET)
From: Andreas Barth <[email-masked]>
Reply-To: [email-masked]
To: Brad Templeton <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Wed, 17 Mar 1999, Brad Templeton wrote:
> The question is, what percentage of news sites are confirming newgroup
> with pgpverify?  I haven't tracked in a while, but back when I was
> using it (1.5 years ago) the answer was few.
I haven't tracked it worldwide, but here in Munich it's about 80% of the 
"important" news sites.

Andi

Date: Wed, 17 Mar 1999 09:00:43 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Wed, Mar 17, 1999 at 11:54:40AM -0500, [email-masked] wrote:
> > 
> > I know in mail the Received header gets simply done multiple times.  Is
> > there any feeling each way?
> 
> Definitely the latter.
> 
> > There is no need to "sign" a "signed" header.   Can you tell me why you
> > would want to do this?   A "signed" header (and any certificate) is
> > verifiable on its own when paired with the signed headers and body.  I
> 
> I'm in the office, and my copy of Schneieder's "Applied Cruptography" is
> at home, but I do seem to recall there being a discussion of signing of
> signatures being important for notary services and non-repudiation schemes.
> 
> Basically, you're signing with *your* key, and your key has some sort
> of chain of trust attached to it so the recipient can verify it was indeed
> your key.  However, for timestamping services and the like, you need to
> get a *second* trusted signature to verify that it was, in fact, done
> at the actual time it claims, and so on.
> 

No, the standard proposed a complex mechanism of self-signing parts of
the signed header (a canonicalized expansion of the header list, and
all the other parts except the signature)

This turns out to be not needed.   You don't have to sign any relevant
part of the Signed header yourself, since any attempt to modify it will
invalidate the signature.   Unless you have things like comments in the
header, which could be modified without invalidating.  However, the ability
to protect comments is not worth the complexity of self-signing.

I strongly advocate a much much simpler canonicalization algorithm along
the lines I outlined.   Take the header list, canonicalize only the
FWS, sort, and concat to hash.    

You are referring above to having a certificate, which is where
somebody *else* signs your key and attributes about your key, and that is
of course important.

Date: Wed, 17 Mar 1999 08:55:59 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

On Wed, Mar 17, 1999 at 05:33:40PM +0100, Andreas Barth wrote:
> On Tue, 16 Mar 1999, Brad Templeton wrote:
> 
> > >   An existing experimental protocol "pgpverify" [PGPVERIFY] is
> 
> > How widespread is it?  We used it but really didn't find a significant
> > population listening to it.  Has that changed of late?
> 
> It may depend on the country, eg. most (or even all, I'm not sure) 
> newsserver here in Germany can use it w/o paying fees to anyone. Most 
> newsservers I know of here in Munich use it.
> 
> The same may also be valid for the US after the patent expires.

It's not the patent issue, you can get a legal pgp.

The question is, what percentage of news sites are confirming newgroup
with pgpverify?  I haven't tracked in a while, but back when I was
using it (1.5 years ago) the answer was few.

To: Brad Templeton <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers 
From: [email-masked]
Date: Wed, 17 Mar 1999 11:54:40 -0500
Sender: [email-masked]

--==_Exmh_990748256P
Content-Type: text/plain; charset=us-ascii

On Tue, 16 Mar 1999 20:55:17 PST, Brad Templeton said:
> What is thought on whether multiple headers should be:
> 
> 	Signed-1:
> 	Signed-2:
> 
> vs.
> 	Signed: l=1;
> 	Signed: l=2;
> 
> I know in mail the Received header gets simply done multiple times.  Is
> there any feeling each way?

Definitely the latter.

> There is no need to "sign" a "signed" header.   Can you tell me why you
> would want to do this?   A "signed" header (and any certificate) is
> verifiable on its own when paired with the signed headers and body.  I

I'm in the office, and my copy of Schneieder's "Applied Cruptography" is
at home, but I do seem to recall there being a discussion of signing of
signatures being important for notary services and non-repudiation schemes.

Basically, you're signing with *your* key, and your key has some sort
of chain of trust attached to it so the recipient can verify it was indeed
your key.  However, for timestamping services and the like, you need to
get a *second* trusted signature to verify that it was, in fact, done
at the actual time it claims, and so on.

I can't offhand speak to whether Usenet or E-mail *need* support for
timestamping and notary services - but if you *are* going to do this,
it will require a second signature.

-- 
				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

--==_Exmh_990748256P
Content-Type: application/pgp-signature

--==_Exmh_990748256P--

Date: Wed, 17 Mar 1999 17:40:32 +0100
To: [email-masked], Jean-Francois Stenuit <[email-masked]>
CC: [email-masked], [email-masked], [email-masked]
From: Brad Knowles <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Wed, Mar 17, 1999, Andreas Barth
<[email-masked]> wrote:

>To stopp cancel wars just use Cancel Locks. Reference implementation is 
>available by Fluffy.

    Or ignore cancels.  We will be implementing cancel locks, I can
assure you.  However, it's been a little while since I looked at that
stuff, and I don't remember whether it depends on INN >= 2.2 or not.

    However, this isn't really relevant to USEFOR.

-- 
  These are my opinions -- not to be taken as official Skynet policy
 ____________________________________________________________________
|o| Brad Knowles, <[email-masked]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News/mail/FTP Admin   Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o| http://www.skynet.be                     Belgium               |o|
\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/
   Usenet is not the web.  Just because the web handles some things
 poorly is not a good reason to apply those same solutions to Usenet.

Date: Wed, 17 Mar 1999 17:33:40 +0100 (MET)
From: Andreas Barth <[email-masked]>
Reply-To: [email-masked]
To: Brad Templeton <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Tue, 16 Mar 1999, Brad Templeton wrote:

> >   An existing experimental protocol "pgpverify" [PGPVERIFY] is

> How widespread is it?  We used it but really didn't find a significant
> population listening to it.  Has that changed of late?

It may depend on the country, eg. most (or even all, I'm not sure) 
newsserver here in Germany can use it w/o paying fees to anyone. Most 
newsservers I know of here in Munich use it.

The same may also be valid for the US after the patent expires.

Andi

Date: Wed, 17 Mar 1999 08:30:00 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Laurence Lundblade <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 12:11 PM +0000 3/12/99, Charles Lindsey wrote:
>
> I think so. But I doubt proletext would ever be just another format of
> text/plain if/when it got proposed for an RFC. It would presumably be
> text/proletext.
>

I haven't looked at proletext, but will point out that any subtype of 
text pretty much went down in flames when we tried sending some 
samples around. We tried text/paragraph with a format that was 
basically plain. Too many mailers default to treating this as an 
attachment.

The only possibility for something to more or less replace text/plain 
would be something like
  text/plain; format=proletext.

If it's not a parameter-based modification to text/plain, we might as 
well use HTML.

LL

Date: Wed, 17 Mar 1999 17:01:46 +0100 (MET)
From: Andreas Barth <[email-masked]>
Reply-To: [email-masked]
To: Jean-Francois Stenuit <[email-masked]>
cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Tue, 16 Mar 1999, Jean-Francois Stenuit wrote:
> On Tue, 16 Mar 1999, Charles Lindsey wrote:

> > There is a strongly perceived need within the Usenet-Format group for
> > a reliable method for the digital signing of headers. It would, on the
> > face of it, be foolish to devise a mechanism applicable to news that
> > would not work also for email, even though that does indeed make the
> > mechanism more complex.

> IMHO, there should be two different schemes for NNTP and SMTP messages,
> because the reason for signing the header is different.
> 
> For instance, we must be able to put multiple signatures on a usenet
> message (one from the author, another from the administrator of the server
> used for posting the article, and another for the moderator) because this
> signing will be used mainly for cancelling an article. As one does not
> usually cancel a mail message, this is not required for mail.

No. There are more lightweight protocols for cancels like Cancel Locks 
which don't need an public/private infrastructure, but that also can't be 
used as an authentification scheme.

Signatures in news are mainly used for moderation and control messages 
(eg. checkgroups) and there is IMHO no great difference to mail messages.

> For mail applications, I don't see the point in signing the headers, as
> long as the body is signed. But I may have missed a point, since I don't
> follow the ietf-822 discussions.

It may be relevant to block re-injection of a mail message, especially if 
the mail is to a mail2news-Gateway or performs some other action 
automatically.

> Here is also a difference, the need is urgent for usenet (because of the
> ongoing cancel wars and other unauthenticated spam) but less urgent for
> mail, as mail servers around the globe are becoming less prone to spam
> propagation.

To stopp cancel wars just use Cancel Locks. Reference implementation is 
available by Fluffy.

> I find that having a single mailing list for "message format" is a good
> idea.
Yes.

Andi

Date: Tue, 16 Mar 1999 16:44:41 +0100 (MET)
From: Jean-Francois Stenuit <[email-masked]>
To: [email-masked], [email-masked]
cc: [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Tue, 16 Mar 1999, Charles Lindsey wrote:

> There is a strongly perceived need within the Usenet-Format group for
> a reliable method for the digital signing of headers. It would, on the
> face of it, be foolish to devise a mechanism applicable to news that
> would not work also for email, even though that does indeed make the
> mechanism more complex.

IMHO, there should be two different schemes for NNTP and SMTP messages,
because the reason for signing the header is different.

For instance, we must be able to put multiple signatures on a usenet
message (one from the author, another from the administrator of the server
used for posting the article, and another for the moderator) because this
signing will be used mainly for cancelling an article. As one does not
usually cancel a mail message, this is not required for mail.

For mail applications, I don't see the point in signing the headers, as
long as the body is signed. But I may have missed a point, since I don't
follow the ietf-822 discussions.

> And even though there may not be such a strong requirement for this
> facility in email, one can imagine some applications that would find it
> useful.
> 
> The draft that I have prepared therefore tries to solve the problem in
> both environments. However, before we get too bogged down in the details
> of what I am proposing, I suggest we first establish the proper forum
> for this discussion, and the form that it should take. For this, I need
> some advice from the various IETF gurus on these lists.
> 
> 1. In the first place, should I aim it at a Draft Standard, or at an
> Experimental Protocol? Note that we shall eventually want to refer to it
> in the final USEFOR document, which is certainly supposed to be on the
> Standards Track, but it might still be useful (and quicker) to have an
> Experimental Protocol before then.

Here is also a difference, the need is urgent for usenet (because of the
ongoing cancel wars and other unauthenticated spam) but less urgent for
mail, as mail servers around the globe are becoming less prone to spam
propagation.

> 3. In the slightly longer term (and especially if it is to become a
> Draft Standard in due course) do I need to apply to the IETF to have a
> proper working group set up? That seems perhaps a slight overkill, so
> would formal discussion on these two lists in parallel be in order even
> in the longer term?

I find that having a single mailing list for "message format" is a good
idea.

> 4. And finally, please do not tear into the nitty gritty and the
> details just yet. Let us discuss the overall principles first, and the
> mechanisms. Clearly, there is much detail to be sorted out eventually,
> but first things first.

agreed.

Regards,

--
 Opinions expressed above are mine, they have not to be taken as a
 skynet official position.
 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
/Jean-Francois "Jef" Stenuit | BELGACOM-Skynet NV/SA | Solaris 2.x,  \
\Network operations manager  | 124, Rue Col. Bourg   | Linux, NT,    /
/Phone (32)(2) 706-1311      | B-1140 Brussels       | Cisco         \
\Fax   (32)(2) 706-1312      | Belgium               | expert ...    /
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Date: Tue, 16 Mar 1999 20:55:17 -0800
From: Brad Templeton <[email-masked]>
To: [email-masked], [email-masked]
Cc: [email-masked]
Subject: Re: Draft for signed headers
Organization: http://www.templetons.com/brad
Sender: [email-masked]

Here are my notes on this draft.  Some of its basic principles are
sound, but there are some major difficulties as I see it:

To sum up:
	You want to sign the body, or you have no security
	You don't want users to check signatures, they can't due to
		revocation problems.
	You don't want users to check MD5s, they want bad articles to
		not even show up in the overview.  Reject after download
		is not that exciting, and also has security holes to
		denial of service attacks.
	OPENPGP is really not at all a suitable choice for USENET.
	The canonicalization algorithm is far too complex and I have
		grave fears about it working.  Indeed, the risks of
		people getting the canonicalization wrong, at least in
		USENET seem greater than the risks of gateways munging
		articles in a way that they invalidate the sig!
	Verified header needs more thought as to usage.

>   An existing experimental protocol "pgpverify" [PGPVERIFY] is

How widespread is it?  We used it but really didn't find a significant
population listening to it.  Has that changed of late?

>   There also exist protocols for the cryptographic signature of
>   bodies of articles, notably S/Mime and PGP/Mime [RFC2015], and it

Multipart/signed is more likely to have relevance to USENET than these.

>   The bodies of articles, Mime messages and multiparts are not
>   directly included in the Signature. Rather, the intention is that
>   each such body part should have a Content-MD5 (or similar) header
>   computed for it, and that header should then be included in the
>   Signature instead.

While that works, what is the goal?  The only value I can see in this is
that it can save you the trouble to test the MD5 or other hash of the body.
But to encourage non-testing of the body is to invite a security hole.

In what circumstances would we not want to also verify the body?  The only
ones I can think of are control messages where the body is meaningless and
normally blank.
>
>   There is also provision for Verified headers which may be added by
>   agents that have checked a Signed header. Verified headers may
>   themselves be included in further Signed headers; this may be
>   especially useful in the case of gateways which find it necessary
>   to change an article in ways that invalidate an original signature.

If the original signature is invalidated, why propagate it?  It is of
value to nobody.  Anybody who has the original pre-modified headers/body
also will have the old signature. 

The main purpose for having some sort of signature other that the original
is signature collapse, where it is inherent that you remove the original.

Another purpose is a gateway where you want to keep both.  However we
would want to discourage that.  It bloats things.  Better that the
original sig be traceable to a common root where possible.

>   Every effort has been made to ensure that signatures remain
>   verifiable in spite of all reasonable (and even unreasonable)
>   changes to which they may be subjected in transit. These include
>   changes to the Content-Transfer-Encoding of body parts (a principle
>   reason for including them only via the Content-MD5 header), changes
>   in the order of headers and of their layout, and encodings and re-
>   encodings of unusual character sets. This is to be achieved by
>   converting headers into a canonical form before they are signed.
>   New headers, yet to be invented, need provide no problem, and there
>   is no commitment to any particular character set (provided header-
>   names remain in ASCII, as at present).

We debated this quite a bit earlier.   Several people reported that
in the digital signature community, the idea of canonicalization is
looked upon with great skepticism.  It's easy to get it wrong.  The
closer you can get to "hash this byte stream" the better.  That's
the philosophy of multipart/signed.

As for transfer encodings, I think that's simply out of the question.
It means that to check the signature -- and checking the headers without
checking the body should most definitely not be considered checking
the signature, not if the body means anything -- one must decode and
put in canconical form the entire article including body!  It is probably
not appropriate for news transports to do this.   It makes no sense
for news readers to do that -- newsreaders don't want to and probably can't
check signatures due to revocation list issues, only transports can.

>2.1.  Syntax of the Signed header
>
>      Signed         = "Signed" ["-" DIGIT] ":" SPACE
>                       [CFWS] protocol CFWS key-id CFWS header-ref-list
>                       CFWS signature CRLF

What is thought on whether multiple headers should be:

	Signed-1:
	Signed-2:

vs.
	Signed: l=1;
	Signed: l=2;

I know in mail the Received header gets simply done multiple times.  Is
there any feeling each way?

Other than that your syntax is pretty similar to my own, however I
recommend the following general principles:

	a) Avoid positional parameters.  They are less extensible and
	   harder to read and easier to get wrong.

	b) Use named parameters, as in:

		Signed: pro=U; key=keyid; Head=+From; Sig=asfasdfsadfasdf

This way you are in MIME standard form, and can add extra parameters
as needed, and in fact if you want, the protocol and Header list parameters
can have defaults and simply not be present 99% of the time, which saves
space.

>   The key-id identifies the key used to generate the signature in a
>   notation dependent upon the protocol (but commonly "0x" followed by
>   hexadecimal digits). The CFWS following it MAY include a comment
>   containing an identification of the person or entity which created
>   the signature.

My expectation is that key-ids need to be allocated in spaces in a
hierarchy, and in fact should probably follow the syntax of email addresses.

In fact, one can have a default key-ID for a given E-mail address that
contains the E-mail address or is the E-mail address.

Most keys may well be issued by sites, and sites will be given the power
to issue keys within their domains, but not in others.  Using the e-mail
address syntax provides an easy way to lay this out.
>
>   The header-ref "mail-standard" is a macro representing a set of
>   common mail headers that SHOULD normally be included when signing
>   the headers of a mail message, and is defined as the list
>
>      Date, From, Reply-To, To, Cc, In-Reply-To, References, Subject,
>      Keywords, Content-Type, Content-ID
>
>   The header-ref "news-standard" performs the same function for news
>   articles, and is defined as the list
>
>      Date, Newsgroups, Distribution, Message-ID, From, Reply-To,
>      Followup-To, References, Subject, Keywords, Control, Content-Type,
>      Content-ID
>[Well, you can imagine those lists are going to be fiercely argued
>over]

Much like my own syntax so how can I disagree!  Actually the lists need
not have too much arguing because you can add modifiers.  The "right"
list is simply the most common one, to avoid bulking out lines.

Note that it's fine to include headers that are not commonly used, but
which are signed when used.   Also, you said you wanted to include
content-md5.

>
>   (ii) A header-ref of the form "XXXX/<m>" (or "XXXX/<m>/<n>..."),
>        where <m> and <n> are numbers and the current level contains a
>        "Content-Type: multipart/*" header, references the header that
>        would be referenced by "XXXX" alone (or by "XXXX/<n>...") in
>        the <m>th sub-part of that multipart, that sub-part now being
>        regarded as the current level.

I have to say this (and the whole body canonicalization) is way too complex.
The body should be a stream of bytes, and that includes internal MIME
headers.  You should not have to parse MIME to check a signature.  There
should be no references to internal headers.  If you want to sign a
mime sub-part, that's fine, I guess.  Do it within the sub-part.  The code
to do what you propose is too much, and too slow.

Plus, since a MIME central part may be content-transfer-encoded, you 
will need to decode it to check these things.

>
>   The signature of a Signed header is constructed in accordance with
>   a given header-ref-list as follows:

There is no need to "sign" a "signed" header.   Can you tell me why you
would want to do this?   A "signed" header (and any certificate) is
verifiable on its own when paired with the signed headers and body.  I
can check if it is authentic.   If I don't know the key named in the
header and have no certificate for it, what good does it do me to be
told by another signature that I am seeing the original signed header?

The protocol described seems too complex, but that's beside the point.
There is no reason to sign other people's signatures.

There is also no need to "sign" any elements of your own signed header.
Self-signing of signatures is complex and adds no value.  If the header
list, flags, key name or options are tampered with, the signature immediately
is invalidated.

>   The purpose of a Signed header is solely to establish that the
>   headers referenced in it were present in an article when that
>   article passed through the hands of the person or entity that
>   generated the signature (and hence that it did indeed pass through

I tend to use the term "keyholder" here.  It verifies that the holder of
the key did indeed verify or certify what is signed.  Keyholders can be
people, machines, job titles, etc.

>   those hands). It SHOULD NOT be taken as an endorsement of whatever
>   is contained in the body of the article. If the contents of the
>   body require such endorsement, then the body SHOULD be signed
>   separately, for example in accordance with PGP/Mime [RFC2015].
>[Hmm! I expect that paragraph to raise some discussion.]

Quite.  Header and body are one.  It's nice to verify my headers but if
somebody can just replace my body with their own you've lulled me into
a very false sense of security.

>
>   Signatures will typically be generated by the originators of
>   articles (to prove the origin), by moderators of moderated
>   newsgroups (to testify to their Approved header), by managers of
>   mailing lists, and by gateways. They SHOULD NOT be generated by
>   intermediate transports and relayers through which the article
>   might pass. This is intended to be an end-to-end protocol, and
>   signatures SHOULD ONLY be added when new, hitherto unsigned,
>   information is added. Moreover, the set of headers included within
>   the signature SHOULD be no more than is necessary to achieve the
>   security desired.

Agreed, and I would further say:
	a) Gateways only would sign when moving off USENET, and their
	signatures would be removed on any re-entry to USENET.
	b) Moderators often modify bodies, they will tend to just remove
	the author's sig (if it is no longer of any use, which is the case
	if so much as a byte of the original article changes)

	On the other hand moderators that do no more than stick on an
	"Approved" header might only add their own sig.

	c) Injectors generally should not sign if the author signed.  They
	should sign if the author didn't.

>
>        signature. If (as will indeed often be the case) it is
>        required to attest that the body (or sub-part) dispatched

As noted, I think that is always the case, not often, so a design based
on it not being always the case may be in error.

Who checks the content-md5?  Not the reader, that's for sure.  (various
reasons making that impossible were outlined in earlier debate).  So
that means the news server, which in 99% of cases is also the relayer.

So you have every relayer having to canonicalize any
content-transfer-encodings to check the md5.

If there were some way I could see not checking the body, the md5 header
would be great.  But it's false security, which can be worse than no
security in some ways.

I'm still not clear on the purpose of the verified header.   Who will
verify?  What will it mean to do so?   Why would you pass on an article
which failed a check?   It's vital not to do so, since forged articles
can steal message-id space as a clever denial of service attack on
somebody's messsages.   (You have a deamon at their ISP look for their
messages, and you issue fake messages with the same message-ids, feeding
them into other ISPs.   The only thing that stops this is if your signatures
don't match and thus your attempt to use those message-ids is rejected.
that doesn't work if people keep passing on articles that failed a
signature check)

Verification has a useful purpose *outside* the article, in digests.
Digests are lists of several messages, with their hashes and message-ids,
signed by a verifier.    If you trust the verifier it lets you avoid
checking the signatures on articles, you just check the hash.

Verificiation is likely to bulk up articles.  If you know your downstream's
can't check the sig, you might as well remove it.   If you know they can,
why would they want to check yours instead?  Only to gain speed, which is
what the digest does.

If you don't know your downstreams and what they can do, is it wise to
add verified headers?

>        NOTE: The Verified header is also useful in the case that a
>        gateway (or a moderator) makes some change to an article that
>        renders an original Signed header invalid. Such a gateway can
>        therefore certify that the original form of the Signed header
>        had been verified, and can then resign the article (including
>        his added Verified header). Likewise, a site (such as the
>        originator's own server) with a well known public key can
>        verify and resign an article whose originator's public key may
>        be less well known. However, Verified headers SHOULD NOT be
>        added as routine by other intermediate sites.

An invalidated signature is just so many garbage bits.   What this gateway
is doing is re-signing, not verifying.   I suppose a gateway might
have some reason (if it's not on USENET) to pass along an article that
failed a signature test, but within USENET it should not.

>
>   It is normally the business of the reading agent of the ultimate
>   recipient to check the correctness of a Content-MD5 or similar
>   header. Nevertheless, an earlier agent that has added a Verified
>   header and also checked such a Content-MD5 header MAY so indicate
>   by mentioning it in a "hashcheck" parameter (or "hashcheck-failed"
>   as appropriate).

There are a number of reasons not to do it this way:

	a) Why put complexity in the reader when the server can have it
	   with less computational effort?
	b) Readers want bad articles out of the overview.  It is of limited
	   value to get an overview menu including bad articles, to select
	   them, and have your reader download them down just to say,
	   "Sorry, this one's MD5 failed"   I simply don't want the bad
	   article to show up in the overview at all.
>
>   It is a sad fact of life that those implementing agents for
>   handling Netnews and Mail cannot resist the temptation to "improve"
>   articles passed through them by rewriting headers that are thought

And this is one thing that a signature algorithim with only limited
canonicalization can stop cold in its tracks.

Screw around with articles and you stop dead as a relay site.  End of
story.  It's a strong incentive not to do it!

As such, canonicalization should be kept to a minimum, ie. sorting and
perhaps a minimum of FWS modification.

>   Furthermore, in the case of Mail it is often required for the
>   transport protocols to modify articles en route, most notably when
>   articles containing octets with the 8th bit set have to be passed
>   through a channel that permits only 7bit.

Let transports using such channels do it and undo it (at least in news)
Transport encoding should be the province of transports, not the format
itself.

>   It is a further sad fact of life that agents which make such
>   changes are not going to go away just because some standard says
>   so. Therefore, the canonicalization algorithm SHOULD endeavour to

But in fact they will go away.  The moment their downstream site stops
accepting any articles they modify in some improper way, they are effectively
off the net.   This is no different than if they were making more dangerous
changes that caused the articles to vanish for other reasons (like say
removing colons)

>   o    Headers may be re-folded to fit within some preferred overall
>        line length.  This may result in the creation of whitespace
>        where none existed before.

I'm against this one.  Relays should do it.  Do they do it?
>
>   o    Trailing whitespace may be removed, and line endings changed
>        to/from CRLF.

This is OK, it never had a semantic effect under the standard.
>
>   o    Field-names may be converted into some usual canonical form
>        (e.g.  "Mime-Version" into "MIME-Version").

Does somebody do this?
>
>   o    Phrases, or parts thereof, may be converted to or from
>        quoted-strings.

Yikes!
>
>   o    Date-times may be rewritten in some preferred format, or into
>        some preferred timezone.

Death to any relay that does this.  You want signature checkers to
canonicalize dates?  That's asking for trouble, I fear.
>
>   o    Headers with non-ASCII characters may be converted to or from
>        the notation defined in [RFC2047].  Observe that there is no
>        canonical way to do this conversion and it is, moreover,
>        frequently performed in contexts where it is not strictly
>        allowed.
>[Other contributions to this list welcomed.]

Rather I would remove entries and boil the canonicalization to something
I think most people can get right namely:

	a) Collapse all FWS in multi line headers to single NL (010)
	b) Sort resulting vectors in byte order and concatenate
	c) Hash the resulting stream.

Frankly I don't think this will fail with many USENET relays.  And the
ones it will fail with deserve the quick death (and resulting quick
rebirth as their authors scurry because they have to) they will get.

I'm talking USENET gateways.  I know mail gateways play games.  Let them.
For mail, multipart/signed or S/MIME are likely to be used for signature.

By the way, some people have told me that even the above very simple
algorithm is too complex, so I know they won't buy into something like
you've detailed.
>
>[Open PGP is the obvious choice for this, since it is widely available
>and is blessed by the IETF. My only reservation is that is comes with
>a rather poor certification system as compared with, say, SPKI. So
>this choice might yet have to be reviewed.]

Open PGP is far from the obvious choice.  In fact, it's not even
a cryptographic algorithm.   It is a very non-USENET style packed bits
encoding for a variety of algorithms.   In fact, far from declaring it
an obvious choice I would have called it an obvious thing to avoid if you
hadn't come up and endorsed it.   It's certificate system, such as it is,
has nothing to do with what USENET certificates want to certify, except
perhaps the most basic E-mail address.

Plus the certificate problem and signature problem are somewhat 
different.   A signature contains a key name, some flags and a signature,
which is just the output of a some digital sig algorithm.

A certificate contains all those but also the things it certifies, expressed
in some certification language that matches the needs of USENET.

Sadly, SPKI is losing the battle to X.509 identity certificates.  But
those certificates are clearly not right for USENET. They are too large
for one thing.

Right now my thought is to try to push an SPKI certificate system.

For the signature, no system is needed at all.  The signature is just
a number.  A 320 bit number in the case of SHA1.  You just base64
encode it.  That's it.  It's just a single number, why would it need
a "format".

>
>   The stream of octets resulting from the canonicalization algorithm
>   is signed, in binary mode (signature type 0x00), in accordance with
>   Open PGP [RFC2440].

You don't include the result of the hash in the signature.  It's redundant.
You calculate the hash yourself, and then test if the signature matches
the hash.  There is no purpose in having a pre-computed copy of the hash
in the signature.

>
>
>   The output of the algorithm MUST be Ascii-armored [RFC2440], but
>   the Armor Header Line ("BEGIN PGP SIGNATURE"), the Armor Headers
>   (e.g.  "Version:"), the blank line following the Armor Headers, and
>   the Armor Tail ("END PGP SIGNATURE") are to be omitted (thus
>   yielding a sequence of base64 characters). Observe that these
>   characters will include a CRC checksum, which SHOULD be on a
>   separate line from the rest of the signature.

Huh?
>
>   The signature included within the Ascii-armor MAY include
>   certificates as evidence that the signing key has the necessary
>   authorization to sign articles of that nature, but such usage is in
>   general deprecated except between parties that have agreed
>   otherwise or where, for some reason, an unusual signatory is
>   signing and attaches a certificate from the usual signatory.

Oh, so you want to include certificates in the signature, and that's the
purpose of this encoding form.  Is there value to doing this?   Because
signatures are long, I think they need to be one per header, which is why
I proposed an independent certificate header.

Date: Tue, 16 Mar 1999 17:45:33 +0000
To: [email-masked], [email-masked]
From: Paul Overell <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

In article <[email-masked]>, Charles Lindsey
<[email-masked]> writes

Nit-picking on the canonicalization algorithm.

>   1.   The field-name (header-name) at the start of the header is
>        converted to lowercase.
>
>   2.   If the header is unstructured, all instances of FWS are
>        replaced by a single SPACE; otherwise (the header is
>        structured and) all instances of FWS are omitted, except
>        within comments where they are replaced by a single SPACE (the
>        header has now been unfolded into a single line). Any
>        whitespace at the end of the header is removed, and it is
>        ensured that the header ends with a single CRLF.
>
        In RFC2234 space is SP, not SPACE.

>   3.   All instances of DQUOTE (ASCII '"') are removed, except when
>        they occur between properly matched pairs of "<" and ">"
>        (thus, in particular, they are not removed within msg-ids).
>
                "<\">"@domain

        Are these < > properly matched?  Should this be canonicalized to

                <\">@domain

        or

                <\>@domain

        I can see what you are trying to do but I think that there may
        be several pitfalls here. 

>   4.   Any date-time occurring in a Date, Resent-Date or Expires
>        header (but not in any other header) is converted into the
>        number of seconds since the start of January 1st 1970 UTC,
>        expressed as a decimal number without leading zeroes.
>
        As phrased the number of seconds since the start of January 1st
        1970 UTC includes leap seconds.  But this will give software a
        problem: how can it be written to cope with messages in the
        future? as future leap-seconds are undecided.

        Better to exclude leap-seconds, in which case you might need a
        note about what to do with a seconds value of 60.

>   5.   Any sequence of octets of length not more than 75 and not
>        including any SPACE (and hence presumed present in the same
>        line prior to Step 2), and which satisfies the syntax for an
>        encoded-word [RFC2047], and which is not enclosed between
>        properly matched pairs of "<" and ">" is replaced by the
>        sequence of octets obtained by decoding it. This is done
>        irrespective of whether that encoded-word was syntactically
>        allowed to be present at that position in the header according
>        to [RFC2047] or any extension thereof.
>

                (\<)=?utf-8?q?=E2=82=A0?=(\>)

        Are these < > properly matched?  Should this be canonicalized to

                (\<)₠(\>)

        or not?

        (For those without utf-8 it's a euro symbol).

Regards

-- 
Paul Overell                                        T U R N P I K E  Ltd

Date: Tue, 16 Mar 1999 17:45:33 +0000
To: [email-masked], [email-masked]
From: Paul Overell <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

In article <[email-masked]>, Charles Lindsey
<[email-masked]> writes

Nit-picking on the canonicalization algorithm.

>   1.   The field-name (header-name) at the start of the header is
>        converted to lowercase.
>
>   2.   If the header is unstructured, all instances of FWS are
>        replaced by a single SPACE; otherwise (the header is
>        structured and) all instances of FWS are omitted, except
>        within comments where they are replaced by a single SPACE (the
>        header has now been unfolded into a single line). Any
>        whitespace at the end of the header is removed, and it is
>        ensured that the header ends with a single CRLF.
>
        In RFC2234 space is SP, not SPACE.

>   3.   All instances of DQUOTE (ASCII '"') are removed, except when
>        they occur between properly matched pairs of "<" and ">"
>        (thus, in particular, they are not removed within msg-ids).
>
                "<\">"@domain

        Are these < > properly matched?  Should this be canonicalized to

                <\">@domain

        or

                <\>@domain

        I can see what you are trying to do but I think that there may
        be several pitfalls here. 

>   4.   Any date-time occurring in a Date, Resent-Date or Expires
>        header (but not in any other header) is converted into the
>        number of seconds since the start of January 1st 1970 UTC,
>        expressed as a decimal number without leading zeroes.
>
        As phrased the number of seconds since the start of January 1st
        1970 UTC includes leap seconds.  But this will give software a
        problem: how can it be written to cope with messages in the
        future? as future leap-seconds are undecided.

        Better to exclude leap-seconds, in which case you might need a
        note about what to do with a seconds value of 60.

>   5.   Any sequence of octets of length not more than 75 and not
>        including any SPACE (and hence presumed present in the same
>        line prior to Step 2), and which satisfies the syntax for an
>        encoded-word [RFC2047], and which is not enclosed between
>        properly matched pairs of "<" and ">" is replaced by the
>        sequence of octets obtained by decoding it. This is done
>        irrespective of whether that encoded-word was syntactically
>        allowed to be present at that position in the header according
>        to [RFC2047] or any extension thereof.
>

                (\<)=?utf-8?q?=E2=82=A0?=(\>)

        Are these < > properly matched?  Should this be canonicalized to

                (\<)₠(\>)

        or not?

        (For those without utf-8 it's a euro symbol).

Regards

-- 
Paul Overell                                        T U R N P I K E  Ltd

Date: Tue, 16 Mar 1999 11:59:32 -0600
To: [email-masked], [email-masked]
From: Paul Hoffman / IMC <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

I don't see a need for this in the email world. In our many discussions of 
the topic, two themes have always emerged:
- Signing in nice but is rarely needed
- When it is needed, you can encapsulate the message with its headers and 
sign the result

--Paul Hoffman, Director
--Internet Mail Consortium

Date: Tue, 16 Mar 1999 11:59:32 -0600
To: [email-masked], [email-masked]
From: Paul Hoffman / IMC <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

I don't see a need for this in the email world. In our many discussions of 
the topic, two themes have always emerged:
- Signing in nice but is rarely needed
- When it is needed, you can encapsulate the message with its headers and 
sign the result

--Paul Hoffman, Director
--Internet Mail Consortium

Date: Tue, 16 Mar 1999 16:44:41 +0100 (MET)
From: Jean-Francois Stenuit <[email-masked]>
To: [email-masked], [email-masked]
cc: [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Tue, 16 Mar 1999, Charles Lindsey wrote:

> There is a strongly perceived need within the Usenet-Format group for
> a reliable method for the digital signing of headers. It would, on the
> face of it, be foolish to devise a mechanism applicable to news that
> would not work also for email, even though that does indeed make the
> mechanism more complex.

IMHO, there should be two different schemes for NNTP and SMTP messages,
because the reason for signing the header is different.

For instance, we must be able to put multiple signatures on a usenet
message (one from the author, another from the administrator of the server
used for posting the article, and another for the moderator) because this
signing will be used mainly for cancelling an article. As one does not
usually cancel a mail message, this is not required for mail.

For mail applications, I don't see the point in signing the headers, as
long as the body is signed. But I may have missed a point, since I don't
follow the ietf-822 discussions.

> And even though there may not be such a strong requirement for this
> facility in email, one can imagine some applications that would find it
> useful.
> 
> The draft that I have prepared therefore tries to solve the problem in
> both environments. However, before we get too bogged down in the details
> of what I am proposing, I suggest we first establish the proper forum
> for this discussion, and the form that it should take. For this, I need
> some advice from the various IETF gurus on these lists.
> 
> 1. In the first place, should I aim it at a Draft Standard, or at an
> Experimental Protocol? Note that we shall eventually want to refer to it
> in the final USEFOR document, which is certainly supposed to be on the
> Standards Track, but it might still be useful (and quicker) to have an
> Experimental Protocol before then.

Here is also a difference, the need is urgent for usenet (because of the
ongoing cancel wars and other unauthenticated spam) but less urgent for
mail, as mail servers around the globe are becoming less prone to spam
propagation.

> 3. In the slightly longer term (and especially if it is to become a
> Draft Standard in due course) do I need to apply to the IETF to have a
> proper working group set up? That seems perhaps a slight overkill, so
> would formal discussion on these two lists in parallel be in order even
> in the longer term?

I find that having a single mailing list for "message format" is a good
idea.

> 4. And finally, please do not tear into the nitty gritty and the
> details just yet. Let us discuss the overall principles first, and the
> mechanisms. Clearly, there is much detail to be sorted out eventually,
> but first things first.

agreed.

Regards,

--
 Opinions expressed above are mine, they have not to be taken as a
 skynet official position.
 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
/Jean-Francois "Jef" Stenuit | BELGACOM-Skynet NV/SA | Solaris 2.x,  \
\Network operations manager  | 124, Rue Col. Bourg   | Linux, NT,    /
/Phone (32)(2) 706-1311      | B-1140 Brussels       | Cisco         \
\Fax   (32)(2) 706-1312      | Belgium               | expert ...    /
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Date: Tue, 16 Mar 1999 15:16:02 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked], [email-masked]
Subject: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: text/plain; charset=us-ascii
Content-MD5: 5/sktzZqbCARhfqCUnLi/Q==
X-Sun-Data-Type: text
Content-Transfer-Encoding: 7bit

There is a strongly perceived need within the Usenet-Format group for
a reliable method for the digital signing of headers. It would, on the
face of it, be foolish to devise a mechanism applicable to news that
would not work also for email, even though that does indeed make the
mechanism more complex.

And even though there may not be such a strong requirement for this
facility in email, one can imagine some applications that would find it
useful.

The draft that I have prepared therefore tries to solve the problem in
both environments. However, before we get too bogged down in the details
of what I am proposing, I suggest we first establish the proper forum
for this discussion, and the form that it should take. For this, I need
some advice from the various IETF gurus on these lists.

1. In the first place, should I aim it at a Draft Standard, or at an
Experimental Protocol? Note that we shall eventually want to refer to it
in the final USEFOR document, which is certainly supposed to be on the
Standards Track, but it might still be useful (and quicker) to have an
Experimental Protocol before then.

2. For the moment, I am posting this to both the ietf-82 and the
usenet-format lists, with Reply-To set to both, and I request that the
debate continue in BOTH places for now. Would it be useful for me to set
up a mailing list somewhere that automatically posted to both?

3. In the slightly longer term (and especially if it is to become a
Draft Standard in due course) do I need to apply to the IETF to have a
proper working group set up? That seems perhaps a slight overkill, so
would formal discussion on these two lists in parallel be in order even
in the longer term?

4. And finally, please do not tear into the nitty gritty and the
details just yet. Let us discuss the overall principles first, and the
mechanisms. Clearly, there is much detail to be sorted out eventually,
but first things first.

Cc: to David Lawrence (who is on at least one of the lists) because it impinges 
upon his pgpverify.

Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: application/octet-stream; name=signed
Content-MD5: 4S3wGLHhncD0mqi41KTnCA==
Content-Description: signed
Content-Disposition: attachment; filename=signed
Content-Transfer-Encoding: 7bit

Network Working Group                        Charles H. Lindsey
Internet-Draft                               University of Manchester
                                             February 1999

                  Signed Headers in Mail and Netnews

Status of this Memo

   This document is an Internet-Draft. Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its
   areas, and its working groups. Note that other groups may also
   distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet-
   Drafts as reference material or to cite them other than as "work
   in progress."

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe),
   munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or
   ftp.isi.edu (US West Coast).

Abstract

   TBD

1.  Introduction

1.1.  Scope and Objectives

[This is a Draft of a Draft, for discussion within the USEFOR and
ietf-822 mailing lists until the best format for putting it forward
has been decided, in particular whether it should continue under one
or the other of those lists, or whether it needs a separate IETF
Working Group. It also needs to be decided whether it should be aimed
towards the Standards track, or at an Experimental Protocol.]

[Note that comments in square brackets like this one constitute meta-
text - not intended to form part of any final document.]

   "Netnews" is a set of protocols [USEFOR] that enables news
   "articles" to be broadcast to potentially-large audiences, using a
   flooding algorithm which propagates copies throughout a network of
   participating hosts. The huge growth in the use of this protocol in
   recent years has been accompanied by many attempts to abuse the
   system by causing it to appear that articles came from parties
   other than those that actually injected them, or that they had been
   posted with some Approval that the real poster was not entitled to
   give, or that they otherwise appeared to be different from what
   they actually were. The effects of such abuse are particularly
   accute in the case of "Control" articles which can cause newsgroups
   to be created or removed on hosts worldwide, or which can cause
   unauthorized deletion of articles already received and stored on
   such hosts. It is therefore considered essential to provide a
   cryptographically secure means whereby it can be established beyond
   doubt that the source and structure of articles are exactly as they
   purport to be.

   "Electronic Mail" is a system for routing "messages" [MESSFOR]
   between individual computer users, usually on a one-to-one basis.
   The formats of Mail messages and News articles have deliberately
   been made to be similar, so that messages may be gatewayed to news
   systems and vice-versa. Whilst the scope for an abuser to cause
   serious damage is smaller in the case of Mail as opposed to News
   (because of the absence of Control headers), it would still be an
   added safeguard to be able to secure Mail headers
   cryptographically, especially for mail circulated via a mailing
   list. Moreover, the possibility of gatewaying between news and mail
   make it essential that the same security techniques should work in
   both.

   An existing experimental protocol "pgpverify" [PGPVERIFY] is
   already in widespread use for authenticating Control messages for
   creating and removing newsgroups within Usenet, and has proven
   itself very successful in mitigating the effects of malicious
   attacks against the integrity of Usenet.  This present standard is
   largely based upon pgpverify; however, pgpverify is unsuitable for
   more widespread use as it stands because it is unable to cope with
   folded headers and with the changes a mail message is likely to
   undergo during transport. A second similar experimental protocol
   "pgpmoose" [PGPMOOSE] is also currently in use for protecting
   moderated newsgroups against unauthorized postings.

   There also exist protocols for the cryptographic signature of
   bodies of articles, notably S/Mime and PGP/Mime [RFC2015], and it
   is moreover common to sign such bodies using PGP alone without the
   use of Mime [RFC2045] et seq at all. However, these protocols
   cannot, by their nature, be used to sign headers. Moreover, since
   the signature is applied after any Content-Transfer-Encoding
   [RFC2045], it may be impossible to verify the signature if the
   Content-Transfer-Encoding should be changed as the message passes
   through a succession of sites during transport. Nevertheless, this
   present standard does not attempt to usurp those protocols, but
   merely provides the means to sign headers, both of complete
   messages and of headers embedded in Mime messages and multiparts.

[This document has been designed to fit on top of the drafts currently
in preparation for Mail [MESSFOR] and for News [USEFOR].  It is
expected that at least the Mail draft will have progressed to the RFC
stage by the time the present document is complete, at which time all
references to [MESSFOR] in the present text will be replaced by
references to that RFC. If it is thought wise to issue this document
before [USEFOR] is complete, then that reference will have to be to
[RFC1036] instead.]

1.2.  Notations and Conventions

1.2.1.  Requirements notation

   Certain words used to define the significance of individual
   requirements are capitalized. "MUST", "SHOULD", "MAY" and the same
   words followed by "NOT" should be read as having the same meaning
   as in [RFC2119].  In particular, to be fully compliant with this
   document, software must satisfy every relevant "MUST" requirement.
   Software that satisfies every relevant "SHOULD" requirement but not
   every "MUST" requirement is partially compliant.

1.2.2.  Syntactic notation

   This document uses the Augmented Backus Naur Form described in
   [RFC2234].  A discussion of this is outside the bounds of this
   document, but it is expected that implementors will be able to
   quickly understand it with reference to the defining document.

1.3.  Overview

   This standard makes provision for Signed headers to be included in
   mail and news articles and in Mime messages and multiparts. A
   Signed header provides a cryptographic signature over a named set
   of other headers, including lower level headers contained in Mime
   messages and multiparts below the current level. Such signatures
   can give assurance to a recipient who verifies them that those
   headers have not been changed or added to in transit, and/or that
   the article was indeed sent by its purported originator.

   The bodies of articles, Mime messages and multiparts are not
   directly included in the Signature. Rather, the intention is that
   each such body part should have a Content-MD5 (or similar) header
   computed for it, and that header should then be included in the
   Signature instead.

   There is also provision for Verified headers which may be added by
   agents that have checked a Signed header. Verified headers may
   themselves be included in further Signed headers; this may be
   especially useful in the case of gateways which find it necessary
   to change an article in ways that invalidate an original signature.

   Every effort has been made to ensure that signatures remain
   verifiable in spite of all reasonable (and even unreasonable)
   changes to which they may be subjected in transit. These include
   changes to the Content-Transfer-Encoding of body parts (a principle
   reason for including them only via the Content-MD5 header), changes
   in the order of headers and of their layout, and encodings and re-
   encodings of unusual character sets. This is to be achieved by
   converting headers into a canonical form before they are signed.
   New headers, yet to be invented, need provide no problem, and there
   is no commitment to any particular character set (provided header-
   names remain in ASCII, as at present).

   Provision is made for different protocols which may be required in
   the future.  However, this standard defines just one, recommended
   protocol, and it is not desirable that other protocols should be
   defined unless and until serious deficiencies in the existing ones
   have been revealed.

2.  Basic Structure of Authenticating Headers

   A Signed or a Verified header may appear in the headers of a mail
   message or a news article, or in the headers of a Mime multipart
   sub-part or of a Mime message/rfc822 object (or indeed of any
   similar Mime object yet to be invented). In all cases, the term
   "current level" encompasses the entire set of headers in that same
   object. Where the headers at the current level include a "Content-
   Type:  multipart/*" or "Content-Type:  message/*" header, lower-
   level headers can arise within its sub-parts.

2.1.  Syntax of the Signed header

      Signed         = "Signed" ["-" DIGIT] ":" SPACE
                       [CFWS] protocol CFWS key-id CFWS header-ref-list
                       CFWS signature CRLF
      protocol       = ietf-token / x-token
      key-id         = token
      header-ref-list= header-ref *( [CFWS] "," [CFWS] header-ref )
      header-ref     = [ "+" / "-" ] ( field-name *( "/" 1*DIGIT )
                       / "mail-standard" / "news-standard" )
      field-name     ; <see [MESSFOR]>
      CFWS           ; <see [MESSFOR]>
      FWS            ; <see [MESSFOR]>
      ietf-token     = <An extension token defined by a standards-track
                       or experimental protocol RFC and registered
                       with IANA>
      x-token        = The two characters "X-" or "x-" followed, with
                       no intervening white space, by any token>
      token          = 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                       or tspecials> [CFWS]
      tspecials      = "(" / ")" / "<" / ">" / "@" /
                       "," / ";" / ":" / "\" / DQUOTE /
                       "/" / "[" / "]" / "?" / "="
      signature      = 1*( btext [FWS] )
      btext          = %x41-5A / %x61-7A / %x30-39 / "+" / "/" / "="
                       ; base 64 chars

        NOTE: A signature may end in FWS, but not in CFWS. The
        requirement for an explicit SPACE after the ":" is to ensure
        compatibility with the syntax of Netnews [USEFOR]; it is not
        strictly necessary for Mail.

   The use of a DIGIT in the Signed header allows for 10 distinct such
   headers at any one level. "Signed:" is synonymous with "Signed-0:".
   This is more than sufficient for the intended usage (it would be
   most unusual to get beyond Signed-2) whilst still permitting
   implementations to check header-names against a fixed list of valid
   names. There MUST NOT be more than one Signed header with the same
   DIGIT within one set of headers.

   Tokens used for key-ids are case-insensitive.  "Foobar" is the
   preferred protocol defined by this standard.  It is desirable to
   keep the number of recognized protocols to an absolute minimum, and
   it is anticipated that further protocols would only be needed in
   the event that serious cryptographic deficiencies were to be found
   in the existing ones.
[Obviously, "foobar" is just a placeholder for whatever name is
finally chosen.]

   The key-id identifies the key used to generate the signature in a
   notation dependent upon the protocol (but commonly "0x" followed by
   hexadecimal digits). The CFWS following it MAY include a comment
   containing an identification of the person or entity which created
   the signature.

   The header-ref-list indicates those header-refs, at or below the
   current level, which are covered by the signature. The ordering of
   this list is significant. A header-ref prefixed by a "+", or not
   prefixed at all, indicates a header-ref to be added to the list
   defined by those preceding it, and a header-ref prefixed by "-"
   indicates a header-ref to be removed from the header-refs defined
   by the list preceding it.

   The header-ref "mail-standard" is a macro representing a set of
   common mail headers that SHOULD normally be included when signing
   the headers of a mail message, and is defined as the list

      Date, From, Reply-To, To, Cc, In-Reply-To, References, Subject,
      Keywords, Content-Type, Content-ID

   The header-ref "news-standard" performs the same function for news
   articles, and is defined as the list

      Date, Newsgroups, Distribution, Message-ID, From, Reply-To,
      Followup-To, References, Subject, Keywords, Control, Content-Type,
      Content-ID
[Well, you can imagine those lists are going to be fiercely argued
over]

        NOTE: Those lists have carefully excluded those headers (such
        as Sender and Content-Transfer-Encoding) which are liable to
        be added or altered by sites downstream from the one which
        generated the Signed header. If some header-ref in the list
        matches no header in the actual article, then it comprises an
        assertion that no such header was present when the article was
        signed. Headers which are routinely added to or altered as the
        article progresses through transports (such as Path, Received
        and Xref) SHOULD NOT be included in a header-ref-list, and
        neither should any header which appears twice in the set of
        headers. A header-ref prefixed by "-" may be used to exclude
        any header-ref from one of the standard lists.

2.2.  Semantics of the Signed header

   Where the headers at the current level include a "Content-Type:
   multipart/*" or "Content-Type: message/*" header, lower-level
   headers within its sub-parts may be referenced as follows:

   (i)  A header-ref not postfixed by any "/ DIGIT"s references the
        header of that name, if any, at the current level. Header-refs
        are, for this purpose, considered as case-insensitive.

   (ii) A header-ref of the form "XXXX/<m>" (or "XXXX/<m>/<n>..."),
        where <m> and <n> are numbers and the current level contains a
        "Content-Type: multipart/*" header, references the header that
        would be referenced by "XXXX" alone (or by "XXXX/<n>...") in
        the <m>th sub-part of that multipart, that sub-part now being
        regarded as the current level.

   (iii)A header-ref of the form "XXXX/1", where the current level
        contains a "Content-Type: message/rfc822" header (or any other
        message type which provides for its own set of headers),
        references the header that would be referenced by "XXXX" alone
        in that message object.

   (iv) A header-ref that does not match up with multipart or message
        Content-Type headers as indicated above MUST NOT be used.

   (v)  For example "Content-MD5/3/2" references the Content-MD5
        header of the second part of a multipart, which is itself the
        third part of a multipart established at the current level.

   A protocol, as established by this standard or by any RFC which
   extends this standard, comprises two parts: a "canonicalization
   algorithm" and a "cryptographic algorithm".

   The signature of a Signed header is constructed in accordance with
   a given header-ref-list as follows:

   1.   A partial Signed header is constructed from the name of the
        protocol, the key-id identifying the cryptographic key used
        (possibly followed by a comment indicating the person or
        entity responsible), followed by the header-ref-list, its
        following CFWS and a CRLF.

   2.   The header-ref-list is reduced by expanding the macros "mail-
        standard" and/or "news-standard", removing from the preceding
        part of the list any header-ref prefixed by a "-", and
        removing any duplicates.

   3.   The partial Signed header followed by all the headers
        referenced by the reduced header-ref-list (being headers at
        the current level or encapsulated within multiparts at any
        lower level) are concatenated to produce a list of headers to
        be signed.

   4.   The list of headers to be signed is subjected to the
        canonicalization algorithm of the protocol to produce a
        canonicalized list.

   5.   The canonicalized list is subjected to the cryptographic
        algorithm of the protocol to produce an octet stream
        representing the signature.

   6.   If the octet stream as produced by the cryptographic algorithm
        is not already in the form of base64 characters, it is now
        encoded in base64 [RFC2045].  It is then split into convenient
        lines by the insertion of FWS, and appended to the partial
        Signed header after a FWS.

   7.   The Signed header thus constructed is then incorporated into
        the set of headers at the current level.

   The signature of a Signed header is verified as follows:

   1.   The signature is removed from the Signed header to give a
        partial Signed header.

   2-4. The corresponding steps of the process that constructed the
        header are taken, producing a canonicalized list.

   5.   The public key identified by the key-id is now used by the
        cryptographic algorithm of the protocol to verify the
        signature. This may result in a simple pass-fail, or it may
        return some indication of the privileges (such as the
        authority to issue certain news control messages or to manage
        some mailing list) enjoyed by the owner of that key.

   The purpose of a Signed header is solely to establish that the
   headers referenced in it were present in an article when that
   article passed through the hands of the person or entity that
   generated the signature (and hence that it did indeed pass through
   those hands). It SHOULD NOT be taken as an endorsement of whatever
   is contained in the body of the article. If the contents of the
   body require such endorsement, then the body SHOULD be signed
   separately, for example in accordance with PGP/Mime [RFC2015].
[Hmm! I expect that paragraph to raise some discussion.]

   Signatures will typically be generated by the originators of
   articles (to prove the origin), by moderators of moderated
   newsgroups (to testify to their Approved header), by managers of
   mailing lists, and by gateways. They SHOULD NOT be generated by
   intermediate transports and relayers through which the article
   might pass. This is intended to be an end-to-end protocol, and
   signatures SHOULD ONLY be added when new, hitherto unsigned,
   information is added. Moreover, the set of headers included within
   the signature SHOULD be no more than is necessary to achieve the
   security desired.

        NOTE: It will be observed that no provision has been made to
        include the bodies of an article or of its sub-parts in the
        signature. If (as will indeed often be the case) it is
        required to attest that the body (or sub-part) dispatched
        along with the set of headers is the same as the body that was
        delivered at the far end, then the proper procedure is to
        construct a Content-MD5 header [RFC1864] for that body (or
        sub-part) and to include that Content-MD5 amongst the headers
        that are signed. Doing it this way confers three advantages:
        a) The Content-MD5 header is constructed in such a way that it
        is immune to changes of Content-Transfer-Encoding to which an
        article, or its sub-parts, may be subjected during transport.
        b) Given that many user agents already routinely construct a
        Content-MD5 header, and verify it on receipt (a practice much
        to be commended), it should be possible to generate a Signed
        header without an extra pass through the entire body
        (especially in the common case where there are no sub-parts).
        This applies particularly in the case of additional signatures
        by moderators or mailing list managers, who may not need to
        examine the body at all.
        c) If a Content-MD5 header should fail to verify (perhaps
        because of some transmission error) the verification of a
        Signed header might still succeed, giving the recipient at
        least some partial information as to where any problem might
        lie.

        NOTE: If, at some future time, a Content-SHA1 header (or any
        similar header based upon a different hashing algorithm)
        should be invented, it could equally well be used for this
        purpose.

2.3.  Syntax of the Verified header

      Verified       = "Verified" ["-" DIGIT] ":" SPACE
                       [CFWS] [ "FAILED" CFWS ] name-addr
                       *( [CFWS] ";" [CFWS] parameter ) CRLF
      name-addr      ; <see [MESSFOR]>
      parameter      = ( "hashcheck" / "hashcheck-failed" )
                       [CFWS] "=" [CFWS]
                       ( header-ref / DQUOTE header-ref-list DQUOTE ) /
                       attribute "=" [CFWS] value
      attribute      = token
      value          = token / quoted-string
      quoted-string  ; <see [MESSFOR]>

   The use of a DIGIT in the Verified header allows for 10 distinct
   such headers in one article (or one set of headers at some lower
   level). "Verified:" is synonymous with "Verified-0:". Each Verified
   header MUST match some Signed header with the same DIGIT in that
   same set of headers. There MAY be more than one Verified header
   with the same DIGIT within one set of headers (but observe that it
   would not then be possible to include those headers in a further
   Signed header).

   Tokens used for attributes are case-insensitive. The only
   parameters defined by this standard are the "hashcheck" and
   "hashcheck-failed" parameters. Other parameters permitted by the
   syntax are for the purpose of future extensions to this standard,
   and should be ignored except as defined in such extensions.  The
   "value" part of a "hashcheck" parameter covering several header-
   refs is of the form of a quoted-string because in general it can
   contain "," characters (which are tspecials). Even a single
   header-ref MUST NOT be used without DQUOTEs if it contains any
   tspecial character (which is unlikely, but not forbidden by the
   syntax of [MESSFOR]).
[Do we also want a "confidence" parameter for the verifier to express
his certainty of the identity of the original Signer, and if so, what
notation to use?]

   The header-refs following any "hashcheck=" or "hashcheck-failed="
   MUST ONLY be of the form "Content-MD5", or of some other header
   based upon another hashing algorithm which may be invented in the
   future.

2.4.  Semantics of the Verified header

   The Verified header is intended to be added to an article by an
   agent through which the article passes, and serves as an assertion
   that the corresponding Signed header has been cryptographically
   verified by the person or entity identified in the name-addr (or
   otherwise if the optional "FAILED" is present). The addr-spec
   contained in that name-addr MUST be a valid email address by which
   that person or entity may be contacted. The original Signed header
   MUST NOT be removed from the article. The Verified header
   (supposing it is the only one present with that particular DIGIT)
   MAY itself be included in a further Signed header added at the same
   time.

        NOTE: The purpose of a Verified header is to save the ultimate
        recipient the trouble of verifying the cryptographic signature
        himself (which can be time consuming, and may require
        knowledge of public keys not in his possession).  Such a
        verification, if performed close to the ultimate recipient
        (such as by the news or mail server to which he connects)
        could normally be regarded as adequate evidence of
        authenticity, even if not signed itself. It would be hard
        (certainly in the case of news) for a malicious interloper to
        cause such a verification to appear bearing the identity of
        the local server of each ultimate recipient.

        NOTE: The Verified header is also useful in the case that a
        gateway (or a moderator) makes some change to an article that
        renders an original Signed header invalid. Such a gateway can
        therefore certify that the original form of the Signed header
        had been verified, and can then resign the article (including
        his added Verified header). Likewise, a site (such as the
        originator's own server) with a well known public key can
        verify and resign an article whose originator's public key may
        be less well known. However, Verified headers SHOULD NOT be
        added as routine by other intermediate sites.

   It is normally the business of the reading agent of the ultimate
   recipient to check the correctness of a Content-MD5 or similar
   header. Nevertheless, an earlier agent that has added a Verified
   header and also checked such a Content-MD5 header MAY so indicate
   by mentioning it in a "hashcheck" parameter (or "hashcheck-failed"
   as appropriate).

3.  Protocol definition

3.1.  Requirements for canonicalization algorithms

   It is a sad fact of life that those implementing agents for
   handling Netnews and Mail cannot resist the temptation to "improve"
   articles passed through them by rewriting headers that are thought
   not to conform to some real or supposed standard. Experience shows
   that, in the majority of cases, such tinkering makes matters worse
   rather than better, and for that reason [USEFOR] and, to a lesser
   extent, [MESSFOR] and [SMTP] try to forbid it, especially when
   perpetrated by relaying and transport agents (there are arguments
   in favour of allowing injecting agents and other agents close to
   the originator to do some limited cleanups, especially where it is
   impractical to return the article to the originator for
   correction).

   Furthermore, in the case of Mail it is often required for the
   transport protocols to modify articles en route, most notably when
   articles containing octets with the 8th bit set have to be passed
   through a channel that permits only 7bit.

   It is a further sad fact of life that agents which make such
   changes are not going to go away just because some standard says
   so. Therefore, the canonicalization algorithm SHOULD endeavour to
   enable the headers of articles to be signed and verified in
   accordance with this standard in spite of such tinkerings, insofar
   as they can be anticipated. The following list indicates some
   common practices which are worth detecting and protecting against.

   o    Headers may be re-folded to fit within some preferred overall
        line length.  This may result in the creation of whitespace
        where none existed before.

   o    Trailing whitespace may be removed, and line endings changed
        to/from CRLF.

   o    Field-names may be converted into some usual canonical form
        (e.g.  "Mime-Version" into "MIME-Version").

   o    Phrases, or parts thereof, may be converted to or from
        quoted-strings.

   o    Date-times may be rewritten in some preferred format, or into
        some preferred timezone.

   o    Headers with non-ASCII characters may be converted to or from
        the notation defined in [RFC2047].  Observe that there is no
        canonical way to do this conversion and it is, moreover,
        frequently performed in contexts where it is not strictly
        allowed.
[Other contributions to this list welcomed.]

   Since the slightest change to a canonicalization algorithm will
   render it inoperable with previous versions, such an algorithm MUST
   NOT be changed once it has been defined by this standard, or any
   extension thereof. In the event of some inadequacy being found, it
   would be necessary to devise and standardize a new algorithm, a
   task not to be undertaken lightly. For this reason,
   canonicalization algorithms SHOULD be designed to cope with the
   widest possible range of headers, including those not yet invented.
   Therefore, they SHOULD NOT, so far as possible, rely on the ability
   to parse any particular header.

        NOTE: A canonicalization algorithm is required simply to
        produce an octet stream for submission to the cryptographic
        algorithm. That stream does not have to be human readable, nor
        does it have to be a syntactically-correct header, nor does it
        have to be convertible back into the original header, or into
        any correct header at all. Insofar as many original headers
        can, in principle, be mapped into the same octet stream, this
        in no way reduces the utility of the algorithm, even though it
        might enable conspiracy theorists to imagine, and even
        implement, various sorts of covert channels for use by
        malicious interlopers.

3.2.  The Foobar protocol

[Suggestions for a proper name on a postcard, please, to /dev/null for
now.]

   The "foobar" protocol is comprised of a canonicalization algorithm
   "foo" and a cryptographic algorithm "bar".

3.2.1.  The Foo canonicalization algorithm

   For the purposes of this algorithm, the headers Subject, Comments,
   Organization and Summary, and all headers starting with "X-", are
   to be considered "unstructured" and all other headers "structured"
   (whether or not they were so described in any other standard). Each
   header to be included in the algorithm, which will in general
   consist of several lines (those after the first commencing with
   whitespace), is processed as follows:

   1.   The field-name (header-name) at the start of the header is
        converted to lowercase.

   2.   If the header is unstructured, all instances of FWS are
        replaced by a single SPACE; otherwise (the header is
        structured and) all instances of FWS are omitted, except
        within comments where they are replaced by a single SPACE (the
        header has now been unfolded into a single line). Any
        whitespace at the end of the header is removed, and it is
        ensured that the header ends with a single CRLF.

   3.   All instances of DQUOTE (ASCII '"') are removed, except when
        they occur between properly matched pairs of "<" and ">"
        (thus, in particular, they are not removed within msg-ids).

   4.   Any date-time occurring in a Date, Resent-Date or Expires
        header (but not in any other header) is converted into the
        number of seconds since the start of January 1st 1970 UTC,
        expressed as a decimal number without leading zeroes.

   5.   Any sequence of octets of length not more than 75 and not
        including any SPACE (and hence presumed present in the same
        line prior to Step 2), and which satisfies the syntax for an
        encoded-word [RFC2047], and which is not enclosed between
        properly matched pairs of "<" and ">" is replaced by the
        sequence of octets obtained by decoding it. This is done
        irrespective of whether that encoded-word was syntactically
        allowed to be present at that position in the header according
        to [RFC2047] or any extension thereof.

        NOTE: The order of these steps is important. The unfolding
        must occur before matched pairs of "<" and ">" can be
        detected. The decoding of supposed encoded-words must take
        place last, because it could produce arbitrary sequences of
        octets (when decoding into UCS-16, for example) which might
        then be confused with ASCII characters such as DQUOTE, etc.
        Whitespace needs to be removed entirely from structured
        headers because it is possible it may have been introduced by
        folding in unexpected places en route, subsequent to the
        original signing.

        NOTE: If an internet mail message in the format defined by
        [MESSFOR] is converted into X.400 mail by a gateway conforming
        to [RFC1327] and then back into internet mail, then it is
        likely that any signature made in accordance with this
        standard will fail to verify. For example comments in headers
        containing addresses (such as in From, Reply-To, etc.) may be
        converted into phrases and moved in front of the addr-spec, or
        even removed entirely, and thus the canonicalized form of the
        message will have been changed.  This old convention for
        storing the Real Name of the person associated with the
        address in a following comment, is now deprecated by both
        [MESSFOR] and [USEFOR], but even where phrases are used for
        this purpose it is possible that other changes to the message
        will still render the signature unverifiable. Note that there
        is in any case no expectation that an internet mail message
        signed according to this standard will ever be able to be
        verified once it has been passed permanently into an X.400
        system, nor vice versa.

[The problems of X.400 gateways gave me much concern. The only case of
interest is where an RFC822 message is bounced off an X.400 mailer (so
it gets converted both ways). This could happen without the parties at
either end being aware of the possibility, so it would be nice if it
worked. However, [RFC1327] is extremely hard to read, and I eventually
has recourse to experimentation with an actual X.400 gateway. Here are
examples of the sorts of mungings encountered, with "before" and
"after" versions of various headers:

From: [email-masked] (Charles H. Lindsey)
From: " (Charles H. Lindsey)" <...>

From: [email-masked] (Charles Lindsey) (Foo Bar)
From: " (Charles Lindsey) (Foo Bar)" <...>

Reply-To: Charles Lindsey <[email-masked]> (Foo Bar)
Reply-To: "Charles Lindsey (Foo Bar)" <...>

These examples show comments after addr-specs being turned to phrases
in front. Removing all comments during canonicalization would not help
here, which is the main reason for dropping that idea.

From: "Charles Lindsey" <[email-masked]>
From: Charles Lindsey <...>

Reply-To: Charles "H." Lindsey <[email-masked]>
Reply-To: "Charles H. Lindsey" <...>

These examples show reasonably intelligent handling of DQUOTE in
phrases, and justify their complete removal in the Foo algorithm.

From: [email-masked]
From: ...

Reply-To: <[email-masked]>
Reply-To: chl <...>

These were trying to establish whether it preserved presence or
absence of <...> around add-specs. It seems it does, but the added
"chl" is worrying.

To: "Autoanswer" <[email-masked]>
        (hopefully)
To: [email-masked]

To: [email-masked]
To: [email-masked]

It seems to throw away all Real Names in To: headers, and to lose the
<...> too.

Cc: [email-masked] (Charles Lindsey)
Cc: ...

And loses all Real Names in Cc:

Message-Id: <[email-masked]> (message)
Message-ID: <[email-masked]>

Comments in Message-IDs get lost. I do not think that matters too
much.

Date: Thu, 4 Mar 1999 19:40:18 GMT (UTC)
Date: Fri, 5 Mar 1999 03:12:54 +0000

It has changed the date when sent to the date when received at the
gateway!  This seems to me a violation of [RFC1327].

Mime-Version: 1.0 (MIME)
Mime-Version: 1.0 (MIME)

It leaves headers it does not need to understand completely alone.

My conclusion is that some messages passed through X.400 gateways are
never going to verify correctly, but that many straightforward ones
will survive with the Foo algorithm as proposed. I think that is the
best that can be done, and better than not doing anything at all.
There is certainly no benefit in removing comments entirely, as I had
once thought there might be.]

3.2.2.  The Bar cryptographic algorithm

[Open PGP is the obvious choice for this, since it is widely available
and is blessed by the IETF. My only reservation is that is comes with
a rather poor certification system as compared with, say, SPKI. So
this choice might yet have to be reviewed.]

   The stream of octets resulting from the canonicalization algorithm
   is signed, in binary mode (signature type 0x00), in accordance with
   Open PGP [RFC2440].

        NOTE: The signature is made in binary mode just in case any
        [RFC2047] decoding into UCS-16 has produced octets which might
        be mistaken for isolated CR, LF or trailing SPACE characters,
        which are treated specially in PGP text mode.

   The output of the algorithm MUST be Ascii-armored [RFC2440], but
   the Armor Header Line ("BEGIN PGP SIGNATURE"), the Armor Headers
   (e.g.  "Version:"), the blank line following the Armor Headers, and
   the Armor Tail ("END PGP SIGNATURE") are to be omitted (thus
   yielding a sequence of base64 characters). Observe that these
   characters will include a CRC checksum, which SHOULD be on a
   separate line from the rest of the signature.

   The signature included within the Ascii-armor MAY include
   certificates as evidence that the signing key has the necessary
   authorization to sign articles of that nature, but such usage is in
   general deprecated except between parties that have agreed
   otherwise or where, for some reason, an unusual signatory is
   signing and attaches a certificate from the usual signatory.

   The signature SHOULD use the DSA public-key algorithm and the SHA-1
   hashing algorithm, and be incorporated in a Version 4 Signature
   Packet in the new format. It MAY alternatively use the combination
   RSA/MD5 with Version 3 in the old format (for compatibility with
   PGP 2.6.x) and it MAY use the combination RSA/SHA-1 with Version 4
   in the new format. Verifiers MUST be able to verify all of these
   forms.

4.  Applications

   It is anticipated that protocols for specific applications of the
   signature mechanisms described in this standard will be devised,
   whether under the auspices of the IETF or otherwise. For example,
   the need to be able to verify the origin of Control messages for
   creating and removing newsgroups and for cancelling articles was a
   prime motivation for creating this standard.

   It is up to each such application to specify appropriate mechanisms
   for establishing a Public Key Infrastructure suited to its purpose.
   Such an infrastructure would provide for the storing, distribution
   and authorization of the necessary public keys (and for revocations
   thereof). This standard establishes no preferred mechanisms in this
   regard, except to draw attention to the possible usefulness of the
   Content-Type application/pgp-keys as defined in [RFC2015].

5.  Examples

[The MD5 hashes and signatures in the following are bogus, but I would
expect to include genuine verifiable ones in the final version.]

5.1.  Newgroup Control message

   A 'newgroup' control message in the format given in [USEFOR].

Newsgroups: comp.foo
From: "Charles Lindsey" <[email-masked]>
Subject: cmsg newgroup comp.foo moderated
Control: newgroup comp.foo moderated
Approved: [email-masked]
Message-ID: <[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Signed: foobar 0x2C15F1A9 (Charles Lindsey)
    news-standard,+content-md5/1,+content-type/1+content-md5/3,
    +content-type/3
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
MIME-Version: 1.0
Content-Type: multipart/news-groupinfo; boundary=88888888

This is a multipart message in MIME format.

--88888888
Content-Type: application news-groupinfo
Content-MD5: T7NtIdVqde62kheQuAHOaw==

For your newsgroups file:
comp.foo                For Foo discussions (Moderated)

--88888888
Content-Type: text/plain

comp.foo a moderated newsgroup which passed its vote for creation
by 424:8 as reported in news.announce.newgroups on 10 Feb 99.

--88888888
Content-Type: application/news-transmission
Content-MD5: +piSsoeNmdin5ukFQuFTlw==

Newsgroups: comp.foo
Path: not-for-relaying
Distribution: local
From: "Charles Lindsey" <[email-masked]>
Message-ID: <919190727.4918/[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Subject: Charter for newsgroup com.foo
Approved: [email-masked]

The charter, culled from the call for votes:

   Comp.foo is a moderated newsgroup for discussing all manner of
   Foos.

Moderation submission address:
   [email-masked]

--88888888--

5.2.  Mail message re-signed by mailing list owner

Resent-From: "Example Mail Server" <[email-masked]>
Verified: [email-masked]; hashcheck=content-md5
Signed-1: FOOBAR 0x2C15F1A9 message-id,date,resent-from,
        verified,signed
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
From: <"[john]"@
    temple.example> (John Smith)
Organization: http://www.temple.example/john
Subject: Submission to mailing list
      in connection with foo.
Date: Sat, 13 Feb 1999 22:59:46 GMT
Content-MD5: +piSsoeNmdin5ukFQuFTlw==
Signed: Foobar 0x2C15F1A9 (John Smith) mail-standard,content-md5
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8

Text of John's message.

--
John's signature.

   Passing this through the foobar canonicalization algorithm produces
   the following, in the case of the "Signed:" header (observe lines
   folded for convenience of this document - the true line endings
   indicated by "CRLF"):

signed:Foobar0x2C15F1A9mail-standard,content-md5CRLF
date:918946786CRLF
from:<"[john]"@temple.example>(John Smith)CRLF
subject: Submission to mailing list in connection with foo.CRLF
content-type:text/plain;charset=us-asciiCRLF
content-md5:+piSsoeNmdin5ukFQuFTlw==CRLF

   And here is the result of canonicalizing the "Signed-1:" header:

signed-1:FOOBAR0x2C15F1A9message-id,date,resent-from,verified,
signedCRLF
message-id:<[email-masked]>CRLF
date:918946786CRLF
resent-from:ExampleMailServer<[email-masked]>CRLF
verified:[email-masked];hashcheck=content-md5CRLF
signed:Foobar0x2C15F1A9mail-standard,content-md5iQCVAwUBNsm8pcJdOtO4ja
nBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501NqacLXEkQFi0I2sWh+9NSW4KdMqN
1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJ
up9lxreI7ASGuUQj6J1AorahY5xtw2ukbEJX/0slc==dTd8CRLF

        NOTE: the second signature signed only that which it had added
        itself, plus sufficient of the original headers to identify
        the original message. It did not need to scan the body to
        recompute the MD5 hash, but effectively included it by signing
        the original "Signed:" header.

6.  Security

   TBD
[What is there to say here?]

7.  References

   [RFC1036] M. Horton and R. Adams, "Standard for Interchange of
        USENET Messages", RFC 1036, December 1987.

   [RFC1327] S. Hardcastle-Kille, "Mapping between X.400(1988) / ISO
        10021 and RFC 822", RFC 1327, May 1992.

   [RFC1864] J. Myers and M. Rose, "The Content-MD5 Header Field", RFC
        1864, October 1995.

   [RFC2015] M. Elkins, "MIME Security with Pretty Good Privacy
        (PGP)", RFC 2015, October 1996.

   [RFC2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
        Extensions (MIME) Part One: Format of Internet Message
        Bodies", RFC 2045, November 1996.

   [RFC2047] K. Moore, "MIME (Multipurpose Internet Mail Extensions)
        Part Three: Message Header Extensions for Non-ASCII Text", RFC
        2047, November 1996.

   [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate
        Requirement Levels", RFC 2119, March 1997.

   [RFC2234] D. Crocker and P. Overell, "Augmented BNF for Syntax
        Specifications: ABNF", RFC 2234, November 1997.

   [RFC2440] J. Callas, L. Donnerhacke, H. Finney, and R. Thayer,
        "OpenPGP Message Format", RFC 2440, November 1998.

   [MESSFOR] P. Resnick, "Internet Message Format Standard", draft-
        ietf-drums-msg-fmt-04.txt, March 1998.

   [PGPMOOSE] Greg Rose, [I need a URL for this], October 1995.

   [PGPVERIFY] David Lawrence,
        ftp://ftp.isc.org/pub/pgpcontrol/README.html.

   [SMTP] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer
        Protocol", draft-ietf-drums-smtpupd-*.txt.

   [USEFOR] Dan Ritter, "News Article Format", draft-ietf-usefor-
        article-format-*.txt.

8.  Acknowledgements

   The author acknowledges the work of David Lawrence, as original
   author of "pgpverify", for many of the ideas contained herein, and
   also many contributions from members of the usenet-format mailing
   list.

9.  Author's address

   Charles Lindsey
   5 Clerewood Avenue, Heald Green, CHEADLE, SK8 3JU, U.K.
   <[email-masked]>

   This draft expires on
   TBD

Appendix A. Model implementation

[It is intended to include an implementation in PERL here.]

Appendix B. Test cases

[It is intended to include some pathological examples here for
implementations to be tested against.]

--7fbd_6533-127c_3cb6-5da4_1972--

Date: Tue, 16 Mar 1999 15:16:02 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked], [email-masked]
Subject: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-MD5: 5/sktzZqbCARhfqCUnLi/Q==
X-Sun-Data-Type: text

There is a strongly perceived need within the Usenet-Format group for
a reliable method for the digital signing of headers. It would, on the
face of it, be foolish to devise a mechanism applicable to news that
would not work also for email, even though that does indeed make the
mechanism more complex.

And even though there may not be such a strong requirement for this
facility in email, one can imagine some applications that would find it
useful.

The draft that I have prepared therefore tries to solve the problem in
both environments. However, before we get too bogged down in the details
of what I am proposing, I suggest we first establish the proper forum
for this discussion, and the form that it should take. For this, I need
some advice from the various IETF gurus on these lists.

1. In the first place, should I aim it at a Draft Standard, or at an
Experimental Protocol? Note that we shall eventually want to refer to it
in the final USEFOR document, which is certainly supposed to be on the
Standards Track, but it might still be useful (and quicker) to have an
Experimental Protocol before then.

2. For the moment, I am posting this to both the ietf-82 and the
usenet-format lists, with Reply-To set to both, and I request that the
debate continue in BOTH places for now. Would it be useful for me to set
up a mailing list somewhere that automatically posted to both?

3. In the slightly longer term (and especially if it is to become a
Draft Standard in due course) do I need to apply to the IETF to have a
proper working group set up? That seems perhaps a slight overkill, so
would formal discussion on these two lists in parallel be in order even
in the longer term?

4. And finally, please do not tear into the nitty gritty and the
details just yet. Let us discuss the overall principles first, and the
mechanisms. Clearly, there is much detail to be sorted out eventually,
but first things first.

Cc: to David Lawrence (who is on at least one of the lists) because it impinges 
upon his pgpverify.

Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: application/octet-stream; name=signed
Content-Transfer-Encoding: 7bit
Content-MD5: 4S3wGLHhncD0mqi41KTnCA==
Content-Description: signed
Content-Disposition: attachment; filename=signed

Network Working Group                        Charles H. Lindsey
Internet-Draft                               University of Manchester
                                             February 1999

                  Signed Headers in Mail and Netnews

Status of this Memo

   This document is an Internet-Draft. Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its
   areas, and its working groups. Note that other groups may also
   distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet-
   Drafts as reference material or to cite them other than as "work
   in progress."

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe),
   munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or
   ftp.isi.edu (US West Coast).

Abstract

   TBD

1.  Introduction

1.1.  Scope and Objectives

[This is a Draft of a Draft, for discussion within the USEFOR and
ietf-822 mailing lists until the best format for putting it forward
has been decided, in particular whether it should continue under one
or the other of those lists, or whether it needs a separate IETF
Working Group. It also needs to be decided whether it should be aimed
towards the Standards track, or at an Experimental Protocol.]

[Note that comments in square brackets like this one constitute meta-
text - not intended to form part of any final document.]

   "Netnews" is a set of protocols [USEFOR] that enables news
   "articles" to be broadcast to potentially-large audiences, using a
   flooding algorithm which propagates copies throughout a network of
   participating hosts. The huge growth in the use of this protocol in
   recent years has been accompanied by many attempts to abuse the
   system by causing it to appear that articles came from parties
   other than those that actually injected them, or that they had been
   posted with some Approval that the real poster was not entitled to
   give, or that they otherwise appeared to be different from what
   they actually were. The effects of such abuse are particularly
   accute in the case of "Control" articles which can cause newsgroups
   to be created or removed on hosts worldwide, or which can cause
   unauthorized deletion of articles already received and stored on
   such hosts. It is therefore considered essential to provide a
   cryptographically secure means whereby it can be established beyond
   doubt that the source and structure of articles are exactly as they
   purport to be.

   "Electronic Mail" is a system for routing "messages" [MESSFOR]
   between individual computer users, usually on a one-to-one basis.
   The formats of Mail messages and News articles have deliberately
   been made to be similar, so that messages may be gatewayed to news
   systems and vice-versa. Whilst the scope for an abuser to cause
   serious damage is smaller in the case of Mail as opposed to News
   (because of the absence of Control headers), it would still be an
   added safeguard to be able to secure Mail headers
   cryptographically, especially for mail circulated via a mailing
   list. Moreover, the possibility of gatewaying between news and mail
   make it essential that the same security techniques should work in
   both.

   An existing experimental protocol "pgpverify" [PGPVERIFY] is
   already in widespread use for authenticating Control messages for
   creating and removing newsgroups within Usenet, and has proven
   itself very successful in mitigating the effects of malicious
   attacks against the integrity of Usenet.  This present standard is
   largely based upon pgpverify; however, pgpverify is unsuitable for
   more widespread use as it stands because it is unable to cope with
   folded headers and with the changes a mail message is likely to
   undergo during transport. A second similar experimental protocol
   "pgpmoose" [PGPMOOSE] is also currently in use for protecting
   moderated newsgroups against unauthorized postings.

   There also exist protocols for the cryptographic signature of
   bodies of articles, notably S/Mime and PGP/Mime [RFC2015], and it
   is moreover common to sign such bodies using PGP alone without the
   use of Mime [RFC2045] et seq at all. However, these protocols
   cannot, by their nature, be used to sign headers. Moreover, since
   the signature is applied after any Content-Transfer-Encoding
   [RFC2045], it may be impossible to verify the signature if the
   Content-Transfer-Encoding should be changed as the message passes
   through a succession of sites during transport. Nevertheless, this
   present standard does not attempt to usurp those protocols, but
   merely provides the means to sign headers, both of complete
   messages and of headers embedded in Mime messages and multiparts.

[This document has been designed to fit on top of the drafts currently
in preparation for Mail [MESSFOR] and for News [USEFOR].  It is
expected that at least the Mail draft will have progressed to the RFC
stage by the time the present document is complete, at which time all
references to [MESSFOR] in the present text will be replaced by
references to that RFC. If it is thought wise to issue this document
before [USEFOR] is complete, then that reference will have to be to
[RFC1036] instead.]

1.2.  Notations and Conventions

1.2.1.  Requirements notation

   Certain words used to define the significance of individual
   requirements are capitalized. "MUST", "SHOULD", "MAY" and the same
   words followed by "NOT" should be read as having the same meaning
   as in [RFC2119].  In particular, to be fully compliant with this
   document, software must satisfy every relevant "MUST" requirement.
   Software that satisfies every relevant "SHOULD" requirement but not
   every "MUST" requirement is partially compliant.

1.2.2.  Syntactic notation

   This document uses the Augmented Backus Naur Form described in
   [RFC2234].  A discussion of this is outside the bounds of this
   document, but it is expected that implementors will be able to
   quickly understand it with reference to the defining document.

1.3.  Overview

   This standard makes provision for Signed headers to be included in
   mail and news articles and in Mime messages and multiparts. A
   Signed header provides a cryptographic signature over a named set
   of other headers, including lower level headers contained in Mime
   messages and multiparts below the current level. Such signatures
   can give assurance to a recipient who verifies them that those
   headers have not been changed or added to in transit, and/or that
   the article was indeed sent by its purported originator.

   The bodies of articles, Mime messages and multiparts are not
   directly included in the Signature. Rather, the intention is that
   each such body part should have a Content-MD5 (or similar) header
   computed for it, and that header should then be included in the
   Signature instead.

   There is also provision for Verified headers which may be added by
   agents that have checked a Signed header. Verified headers may
   themselves be included in further Signed headers; this may be
   especially useful in the case of gateways which find it necessary
   to change an article in ways that invalidate an original signature.

   Every effort has been made to ensure that signatures remain
   verifiable in spite of all reasonable (and even unreasonable)
   changes to which they may be subjected in transit. These include
   changes to the Content-Transfer-Encoding of body parts (a principle
   reason for including them only via the Content-MD5 header), changes
   in the order of headers and of their layout, and encodings and re-
   encodings of unusual character sets. This is to be achieved by
   converting headers into a canonical form before they are signed.
   New headers, yet to be invented, need provide no problem, and there
   is no commitment to any particular character set (provided header-
   names remain in ASCII, as at present).

   Provision is made for different protocols which may be required in
   the future.  However, this standard defines just one, recommended
   protocol, and it is not desirable that other protocols should be
   defined unless and until serious deficiencies in the existing ones
   have been revealed.

2.  Basic Structure of Authenticating Headers

   A Signed or a Verified header may appear in the headers of a mail
   message or a news article, or in the headers of a Mime multipart
   sub-part or of a Mime message/rfc822 object (or indeed of any
   similar Mime object yet to be invented). In all cases, the term
   "current level" encompasses the entire set of headers in that same
   object. Where the headers at the current level include a "Content-
   Type:  multipart/*" or "Content-Type:  message/*" header, lower-
   level headers can arise within its sub-parts.

2.1.  Syntax of the Signed header

      Signed         = "Signed" ["-" DIGIT] ":" SPACE
                       [CFWS] protocol CFWS key-id CFWS header-ref-list
                       CFWS signature CRLF
      protocol       = ietf-token / x-token
      key-id         = token
      header-ref-list= header-ref *( [CFWS] "," [CFWS] header-ref )
      header-ref     = [ "+" / "-" ] ( field-name *( "/" 1*DIGIT )
                       / "mail-standard" / "news-standard" )
      field-name     ; <see [MESSFOR]>
      CFWS           ; <see [MESSFOR]>
      FWS            ; <see [MESSFOR]>
      ietf-token     = <An extension token defined by a standards-track
                       or experimental protocol RFC and registered
                       with IANA>
      x-token        = The two characters "X-" or "x-" followed, with
                       no intervening white space, by any token>
      token          = 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                       or tspecials> [CFWS]
      tspecials      = "(" / ")" / "<" / ">" / "@" /
                       "," / ";" / ":" / "\" / DQUOTE /
                       "/" / "[" / "]" / "?" / "="
      signature      = 1*( btext [FWS] )
      btext          = %x41-5A / %x61-7A / %x30-39 / "+" / "/" / "="
                       ; base 64 chars

        NOTE: A signature may end in FWS, but not in CFWS. The
        requirement for an explicit SPACE after the ":" is to ensure
        compatibility with the syntax of Netnews [USEFOR]; it is not
        strictly necessary for Mail.

   The use of a DIGIT in the Signed header allows for 10 distinct such
   headers at any one level. "Signed:" is synonymous with "Signed-0:".
   This is more than sufficient for the intended usage (it would be
   most unusual to get beyond Signed-2) whilst still permitting
   implementations to check header-names against a fixed list of valid
   names. There MUST NOT be more than one Signed header with the same
   DIGIT within one set of headers.

   Tokens used for key-ids are case-insensitive.  "Foobar" is the
   preferred protocol defined by this standard.  It is desirable to
   keep the number of recognized protocols to an absolute minimum, and
   it is anticipated that further protocols would only be needed in
   the event that serious cryptographic deficiencies were to be found
   in the existing ones.
[Obviously, "foobar" is just a placeholder for whatever name is
finally chosen.]

   The key-id identifies the key used to generate the signature in a
   notation dependent upon the protocol (but commonly "0x" followed by
   hexadecimal digits). The CFWS following it MAY include a comment
   containing an identification of the person or entity which created
   the signature.

   The header-ref-list indicates those header-refs, at or below the
   current level, which are covered by the signature. The ordering of
   this list is significant. A header-ref prefixed by a "+", or not
   prefixed at all, indicates a header-ref to be added to the list
   defined by those preceding it, and a header-ref prefixed by "-"
   indicates a header-ref to be removed from the header-refs defined
   by the list preceding it.

   The header-ref "mail-standard" is a macro representing a set of
   common mail headers that SHOULD normally be included when signing
   the headers of a mail message, and is defined as the list

      Date, From, Reply-To, To, Cc, In-Reply-To, References, Subject,
      Keywords, Content-Type, Content-ID

   The header-ref "news-standard" performs the same function for news
   articles, and is defined as the list

      Date, Newsgroups, Distribution, Message-ID, From, Reply-To,
      Followup-To, References, Subject, Keywords, Control, Content-Type,
      Content-ID
[Well, you can imagine those lists are going to be fiercely argued
over]

        NOTE: Those lists have carefully excluded those headers (such
        as Sender and Content-Transfer-Encoding) which are liable to
        be added or altered by sites downstream from the one which
        generated the Signed header. If some header-ref in the list
        matches no header in the actual article, then it comprises an
        assertion that no such header was present when the article was
        signed. Headers which are routinely added to or altered as the
        article progresses through transports (such as Path, Received
        and Xref) SHOULD NOT be included in a header-ref-list, and
        neither should any header which appears twice in the set of
        headers. A header-ref prefixed by "-" may be used to exclude
        any header-ref from one of the standard lists.

2.2.  Semantics of the Signed header

   Where the headers at the current level include a "Content-Type:
   multipart/*" or "Content-Type: message/*" header, lower-level
   headers within its sub-parts may be referenced as follows:

   (i)  A header-ref not postfixed by any "/ DIGIT"s references the
        header of that name, if any, at the current level. Header-refs
        are, for this purpose, considered as case-insensitive.

   (ii) A header-ref of the form "XXXX/<m>" (or "XXXX/<m>/<n>..."),
        where <m> and <n> are numbers and the current level contains a
        "Content-Type: multipart/*" header, references the header that
        would be referenced by "XXXX" alone (or by "XXXX/<n>...") in
        the <m>th sub-part of that multipart, that sub-part now being
        regarded as the current level.

   (iii)A header-ref of the form "XXXX/1", where the current level
        contains a "Content-Type: message/rfc822" header (or any other
        message type which provides for its own set of headers),
        references the header that would be referenced by "XXXX" alone
        in that message object.

   (iv) A header-ref that does not match up with multipart or message
        Content-Type headers as indicated above MUST NOT be used.

   (v)  For example "Content-MD5/3/2" references the Content-MD5
        header of the second part of a multipart, which is itself the
        third part of a multipart established at the current level.

   A protocol, as established by this standard or by any RFC which
   extends this standard, comprises two parts: a "canonicalization
   algorithm" and a "cryptographic algorithm".

   The signature of a Signed header is constructed in accordance with
   a given header-ref-list as follows:

   1.   A partial Signed header is constructed from the name of the
        protocol, the key-id identifying the cryptographic key used
        (possibly followed by a comment indicating the person or
        entity responsible), followed by the header-ref-list, its
        following CFWS and a CRLF.

   2.   The header-ref-list is reduced by expanding the macros "mail-
        standard" and/or "news-standard", removing from the preceding
        part of the list any header-ref prefixed by a "-", and
        removing any duplicates.

   3.   The partial Signed header followed by all the headers
        referenced by the reduced header-ref-list (being headers at
        the current level or encapsulated within multiparts at any
        lower level) are concatenated to produce a list of headers to
        be signed.

   4.   The list of headers to be signed is subjected to the
        canonicalization algorithm of the protocol to produce a
        canonicalized list.

   5.   The canonicalized list is subjected to the cryptographic
        algorithm of the protocol to produce an octet stream
        representing the signature.

   6.   If the octet stream as produced by the cryptographic algorithm
        is not already in the form of base64 characters, it is now
        encoded in base64 [RFC2045].  It is then split into convenient
        lines by the insertion of FWS, and appended to the partial
        Signed header after a FWS.

   7.   The Signed header thus constructed is then incorporated into
        the set of headers at the current level.

   The signature of a Signed header is verified as follows:

   1.   The signature is removed from the Signed header to give a
        partial Signed header.

   2-4. The corresponding steps of the process that constructed the
        header are taken, producing a canonicalized list.

   5.   The public key identified by the key-id is now used by the
        cryptographic algorithm of the protocol to verify the
        signature. This may result in a simple pass-fail, or it may
        return some indication of the privileges (such as the
        authority to issue certain news control messages or to manage
        some mailing list) enjoyed by the owner of that key.

   The purpose of a Signed header is solely to establish that the
   headers referenced in it were present in an article when that
   article passed through the hands of the person or entity that
   generated the signature (and hence that it did indeed pass through
   those hands). It SHOULD NOT be taken as an endorsement of whatever
   is contained in the body of the article. If the contents of the
   body require such endorsement, then the body SHOULD be signed
   separately, for example in accordance with PGP/Mime [RFC2015].
[Hmm! I expect that paragraph to raise some discussion.]

   Signatures will typically be generated by the originators of
   articles (to prove the origin), by moderators of moderated
   newsgroups (to testify to their Approved header), by managers of
   mailing lists, and by gateways. They SHOULD NOT be generated by
   intermediate transports and relayers through which the article
   might pass. This is intended to be an end-to-end protocol, and
   signatures SHOULD ONLY be added when new, hitherto unsigned,
   information is added. Moreover, the set of headers included within
   the signature SHOULD be no more than is necessary to achieve the
   security desired.

        NOTE: It will be observed that no provision has been made to
        include the bodies of an article or of its sub-parts in the
        signature. If (as will indeed often be the case) it is
        required to attest that the body (or sub-part) dispatched
        along with the set of headers is the same as the body that was
        delivered at the far end, then the proper procedure is to
        construct a Content-MD5 header [RFC1864] for that body (or
        sub-part) and to include that Content-MD5 amongst the headers
        that are signed. Doing it this way confers three advantages:
        a) The Content-MD5 header is constructed in such a way that it
        is immune to changes of Content-Transfer-Encoding to which an
        article, or its sub-parts, may be subjected during transport.
        b) Given that many user agents already routinely construct a
        Content-MD5 header, and verify it on receipt (a practice much
        to be commended), it should be possible to generate a Signed
        header without an extra pass through the entire body
        (especially in the common case where there are no sub-parts).
        This applies particularly in the case of additional signatures
        by moderators or mailing list managers, who may not need to
        examine the body at all.
        c) If a Content-MD5 header should fail to verify (perhaps
        because of some transmission error) the verification of a
        Signed header might still succeed, giving the recipient at
        least some partial information as to where any problem might
        lie.

        NOTE: If, at some future time, a Content-SHA1 header (or any
        similar header based upon a different hashing algorithm)
        should be invented, it could equally well be used for this
        purpose.

2.3.  Syntax of the Verified header

      Verified       = "Verified" ["-" DIGIT] ":" SPACE
                       [CFWS] [ "FAILED" CFWS ] name-addr
                       *( [CFWS] ";" [CFWS] parameter ) CRLF
      name-addr      ; <see [MESSFOR]>
      parameter      = ( "hashcheck" / "hashcheck-failed" )
                       [CFWS] "=" [CFWS]
                       ( header-ref / DQUOTE header-ref-list DQUOTE ) /
                       attribute "=" [CFWS] value
      attribute      = token
      value          = token / quoted-string
      quoted-string  ; <see [MESSFOR]>

   The use of a DIGIT in the Verified header allows for 10 distinct
   such headers in one article (or one set of headers at some lower
   level). "Verified:" is synonymous with "Verified-0:". Each Verified
   header MUST match some Signed header with the same DIGIT in that
   same set of headers. There MAY be more than one Verified header
   with the same DIGIT within one set of headers (but observe that it
   would not then be possible to include those headers in a further
   Signed header).

   Tokens used for attributes are case-insensitive. The only
   parameters defined by this standard are the "hashcheck" and
   "hashcheck-failed" parameters. Other parameters permitted by the
   syntax are for the purpose of future extensions to this standard,
   and should be ignored except as defined in such extensions.  The
   "value" part of a "hashcheck" parameter covering several header-
   refs is of the form of a quoted-string because in general it can
   contain "," characters (which are tspecials). Even a single
   header-ref MUST NOT be used without DQUOTEs if it contains any
   tspecial character (which is unlikely, but not forbidden by the
   syntax of [MESSFOR]).
[Do we also want a "confidence" parameter for the verifier to express
his certainty of the identity of the original Signer, and if so, what
notation to use?]

   The header-refs following any "hashcheck=" or "hashcheck-failed="
   MUST ONLY be of the form "Content-MD5", or of some other header
   based upon another hashing algorithm which may be invented in the
   future.

2.4.  Semantics of the Verified header

   The Verified header is intended to be added to an article by an
   agent through which the article passes, and serves as an assertion
   that the corresponding Signed header has been cryptographically
   verified by the person or entity identified in the name-addr (or
   otherwise if the optional "FAILED" is present). The addr-spec
   contained in that name-addr MUST be a valid email address by which
   that person or entity may be contacted. The original Signed header
   MUST NOT be removed from the article. The Verified header
   (supposing it is the only one present with that particular DIGIT)
   MAY itself be included in a further Signed header added at the same
   time.

        NOTE: The purpose of a Verified header is to save the ultimate
        recipient the trouble of verifying the cryptographic signature
        himself (which can be time consuming, and may require
        knowledge of public keys not in his possession).  Such a
        verification, if performed close to the ultimate recipient
        (such as by the news or mail server to which he connects)
        could normally be regarded as adequate evidence of
        authenticity, even if not signed itself. It would be hard
        (certainly in the case of news) for a malicious interloper to
        cause such a verification to appear bearing the identity of
        the local server of each ultimate recipient.

        NOTE: The Verified header is also useful in the case that a
        gateway (or a moderator) makes some change to an article that
        renders an original Signed header invalid. Such a gateway can
        therefore certify that the original form of the Signed header
        had been verified, and can then resign the article (including
        his added Verified header). Likewise, a site (such as the
        originator's own server) with a well known public key can
        verify and resign an article whose originator's public key may
        be less well known. However, Verified headers SHOULD NOT be
        added as routine by other intermediate sites.

   It is normally the business of the reading agent of the ultimate
   recipient to check the correctness of a Content-MD5 or similar
   header. Nevertheless, an earlier agent that has added a Verified
   header and also checked such a Content-MD5 header MAY so indicate
   by mentioning it in a "hashcheck" parameter (or "hashcheck-failed"
   as appropriate).

3.  Protocol definition

3.1.  Requirements for canonicalization algorithms

   It is a sad fact of life that those implementing agents for
   handling Netnews and Mail cannot resist the temptation to "improve"
   articles passed through them by rewriting headers that are thought
   not to conform to some real or supposed standard. Experience shows
   that, in the majority of cases, such tinkering makes matters worse
   rather than better, and for that reason [USEFOR] and, to a lesser
   extent, [MESSFOR] and [SMTP] try to forbid it, especially when
   perpetrated by relaying and transport agents (there are arguments
   in favour of allowing injecting agents and other agents close to
   the originator to do some limited cleanups, especially where it is
   impractical to return the article to the originator for
   correction).

   Furthermore, in the case of Mail it is often required for the
   transport protocols to modify articles en route, most notably when
   articles containing octets with the 8th bit set have to be passed
   through a channel that permits only 7bit.

   It is a further sad fact of life that agents which make such
   changes are not going to go away just because some standard says
   so. Therefore, the canonicalization algorithm SHOULD endeavour to
   enable the headers of articles to be signed and verified in
   accordance with this standard in spite of such tinkerings, insofar
   as they can be anticipated. The following list indicates some
   common practices which are worth detecting and protecting against.

   o    Headers may be re-folded to fit within some preferred overall
        line length.  This may result in the creation of whitespace
        where none existed before.

   o    Trailing whitespace may be removed, and line endings changed
        to/from CRLF.

   o    Field-names may be converted into some usual canonical form
        (e.g.  "Mime-Version" into "MIME-Version").

   o    Phrases, or parts thereof, may be converted to or from
        quoted-strings.

   o    Date-times may be rewritten in some preferred format, or into
        some preferred timezone.

   o    Headers with non-ASCII characters may be converted to or from
        the notation defined in [RFC2047].  Observe that there is no
        canonical way to do this conversion and it is, moreover,
        frequently performed in contexts where it is not strictly
        allowed.
[Other contributions to this list welcomed.]

   Since the slightest change to a canonicalization algorithm will
   render it inoperable with previous versions, such an algorithm MUST
   NOT be changed once it has been defined by this standard, or any
   extension thereof. In the event of some inadequacy being found, it
   would be necessary to devise and standardize a new algorithm, a
   task not to be undertaken lightly. For this reason,
   canonicalization algorithms SHOULD be designed to cope with the
   widest possible range of headers, including those not yet invented.
   Therefore, they SHOULD NOT, so far as possible, rely on the ability
   to parse any particular header.

        NOTE: A canonicalization algorithm is required simply to
        produce an octet stream for submission to the cryptographic
        algorithm. That stream does not have to be human readable, nor
        does it have to be a syntactically-correct header, nor does it
        have to be convertible back into the original header, or into
        any correct header at all. Insofar as many original headers
        can, in principle, be mapped into the same octet stream, this
        in no way reduces the utility of the algorithm, even though it
        might enable conspiracy theorists to imagine, and even
        implement, various sorts of covert channels for use by
        malicious interlopers.

3.2.  The Foobar protocol

[Suggestions for a proper name on a postcard, please, to /dev/null for
now.]

   The "foobar" protocol is comprised of a canonicalization algorithm
   "foo" and a cryptographic algorithm "bar".

3.2.1.  The Foo canonicalization algorithm

   For the purposes of this algorithm, the headers Subject, Comments,
   Organization and Summary, and all headers starting with "X-", are
   to be considered "unstructured" and all other headers "structured"
   (whether or not they were so described in any other standard). Each
   header to be included in the algorithm, which will in general
   consist of several lines (those after the first commencing with
   whitespace), is processed as follows:

   1.   The field-name (header-name) at the start of the header is
        converted to lowercase.

   2.   If the header is unstructured, all instances of FWS are
        replaced by a single SPACE; otherwise (the header is
        structured and) all instances of FWS are omitted, except
        within comments where they are replaced by a single SPACE (the
        header has now been unfolded into a single line). Any
        whitespace at the end of the header is removed, and it is
        ensured that the header ends with a single CRLF.

   3.   All instances of DQUOTE (ASCII '"') are removed, except when
        they occur between properly matched pairs of "<" and ">"
        (thus, in particular, they are not removed within msg-ids).

   4.   Any date-time occurring in a Date, Resent-Date or Expires
        header (but not in any other header) is converted into the
        number of seconds since the start of January 1st 1970 UTC,
        expressed as a decimal number without leading zeroes.

   5.   Any sequence of octets of length not more than 75 and not
        including any SPACE (and hence presumed present in the same
        line prior to Step 2), and which satisfies the syntax for an
        encoded-word [RFC2047], and which is not enclosed between
        properly matched pairs of "<" and ">" is replaced by the
        sequence of octets obtained by decoding it. This is done
        irrespective of whether that encoded-word was syntactically
        allowed to be present at that position in the header according
        to [RFC2047] or any extension thereof.

        NOTE: The order of these steps is important. The unfolding
        must occur before matched pairs of "<" and ">" can be
        detected. The decoding of supposed encoded-words must take
        place last, because it could produce arbitrary sequences of
        octets (when decoding into UCS-16, for example) which might
        then be confused with ASCII characters such as DQUOTE, etc.
        Whitespace needs to be removed entirely from structured
        headers because it is possible it may have been introduced by
        folding in unexpected places en route, subsequent to the
        original signing.

        NOTE: If an internet mail message in the format defined by
        [MESSFOR] is converted into X.400 mail by a gateway conforming
        to [RFC1327] and then back into internet mail, then it is
        likely that any signature made in accordance with this
        standard will fail to verify. For example comments in headers
        containing addresses (such as in From, Reply-To, etc.) may be
        converted into phrases and moved in front of the addr-spec, or
        even removed entirely, and thus the canonicalized form of the
        message will have been changed.  This old convention for
        storing the Real Name of the person associated with the
        address in a following comment, is now deprecated by both
        [MESSFOR] and [USEFOR], but even where phrases are used for
        this purpose it is possible that other changes to the message
        will still render the signature unverifiable. Note that there
        is in any case no expectation that an internet mail message
        signed according to this standard will ever be able to be
        verified once it has been passed permanently into an X.400
        system, nor vice versa.

[The problems of X.400 gateways gave me much concern. The only case of
interest is where an RFC822 message is bounced off an X.400 mailer (so
it gets converted both ways). This could happen without the parties at
either end being aware of the possibility, so it would be nice if it
worked. However, [RFC1327] is extremely hard to read, and I eventually
has recourse to experimentation with an actual X.400 gateway. Here are
examples of the sorts of mungings encountered, with "before" and
"after" versions of various headers:

From: [email-masked] (Charles H. Lindsey)
From: " (Charles H. Lindsey)" <...>

From: [email-masked] (Charles Lindsey) (Foo Bar)
From: " (Charles Lindsey) (Foo Bar)" <...>

Reply-To: Charles Lindsey <[email-masked]> (Foo Bar)
Reply-To: "Charles Lindsey (Foo Bar)" <...>

These examples show comments after addr-specs being turned to phrases
in front. Removing all comments during canonicalization would not help
here, which is the main reason for dropping that idea.

From: "Charles Lindsey" <[email-masked]>
From: Charles Lindsey <...>

Reply-To: Charles "H." Lindsey <[email-masked]>
Reply-To: "Charles H. Lindsey" <...>

These examples show reasonably intelligent handling of DQUOTE in
phrases, and justify their complete removal in the Foo algorithm.

From: [email-masked]
From: ...

Reply-To: <[email-masked]>
Reply-To: chl <...>

These were trying to establish whether it preserved presence or
absence of <...> around add-specs. It seems it does, but the added
"chl" is worrying.

To: "Autoanswer" <[email-masked]>
        (hopefully)
To: [email-masked]

To: [email-masked]
To: [email-masked]

It seems to throw away all Real Names in To: headers, and to lose the
<...> too.

Cc: [email-masked] (Charles Lindsey)
Cc: ...

And loses all Real Names in Cc:

Message-Id: <[email-masked]> (message)
Message-ID: <[email-masked]>

Comments in Message-IDs get lost. I do not think that matters too
much.

Date: Thu, 4 Mar 1999 19:40:18 GMT (UTC)
Date: Fri, 5 Mar 1999 03:12:54 +0000

It has changed the date when sent to the date when received at the
gateway!  This seems to me a violation of [RFC1327].

Mime-Version: 1.0 (MIME)
Mime-Version: 1.0 (MIME)

It leaves headers it does not need to understand completely alone.

My conclusion is that some messages passed through X.400 gateways are
never going to verify correctly, but that many straightforward ones
will survive with the Foo algorithm as proposed. I think that is the
best that can be done, and better than not doing anything at all.
There is certainly no benefit in removing comments entirely, as I had
once thought there might be.]

3.2.2.  The Bar cryptographic algorithm

[Open PGP is the obvious choice for this, since it is widely available
and is blessed by the IETF. My only reservation is that is comes with
a rather poor certification system as compared with, say, SPKI. So
this choice might yet have to be reviewed.]

   The stream of octets resulting from the canonicalization algorithm
   is signed, in binary mode (signature type 0x00), in accordance with
   Open PGP [RFC2440].

        NOTE: The signature is made in binary mode just in case any
        [RFC2047] decoding into UCS-16 has produced octets which might
        be mistaken for isolated CR, LF or trailing SPACE characters,
        which are treated specially in PGP text mode.

   The output of the algorithm MUST be Ascii-armored [RFC2440], but
   the Armor Header Line ("BEGIN PGP SIGNATURE"), the Armor Headers
   (e.g.  "Version:"), the blank line following the Armor Headers, and
   the Armor Tail ("END PGP SIGNATURE") are to be omitted (thus
   yielding a sequence of base64 characters). Observe that these
   characters will include a CRC checksum, which SHOULD be on a
   separate line from the rest of the signature.

   The signature included within the Ascii-armor MAY include
   certificates as evidence that the signing key has the necessary
   authorization to sign articles of that nature, but such usage is in
   general deprecated except between parties that have agreed
   otherwise or where, for some reason, an unusual signatory is
   signing and attaches a certificate from the usual signatory.

   The signature SHOULD use the DSA public-key algorithm and the SHA-1
   hashing algorithm, and be incorporated in a Version 4 Signature
   Packet in the new format. It MAY alternatively use the combination
   RSA/MD5 with Version 3 in the old format (for compatibility with
   PGP 2.6.x) and it MAY use the combination RSA/SHA-1 with Version 4
   in the new format. Verifiers MUST be able to verify all of these
   forms.

4.  Applications

   It is anticipated that protocols for specific applications of the
   signature mechanisms described in this standard will be devised,
   whether under the auspices of the IETF or otherwise. For example,
   the need to be able to verify the origin of Control messages for
   creating and removing newsgroups and for cancelling articles was a
   prime motivation for creating this standard.

   It is up to each such application to specify appropriate mechanisms
   for establishing a Public Key Infrastructure suited to its purpose.
   Such an infrastructure would provide for the storing, distribution
   and authorization of the necessary public keys (and for revocations
   thereof). This standard establishes no preferred mechanisms in this
   regard, except to draw attention to the possible usefulness of the
   Content-Type application/pgp-keys as defined in [RFC2015].

5.  Examples

[The MD5 hashes and signatures in the following are bogus, but I would
expect to include genuine verifiable ones in the final version.]

5.1.  Newgroup Control message

   A 'newgroup' control message in the format given in [USEFOR].

Newsgroups: comp.foo
From: "Charles Lindsey" <[email-masked]>
Subject: cmsg newgroup comp.foo moderated
Control: newgroup comp.foo moderated
Approved: [email-masked]
Message-ID: <[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Signed: foobar 0x2C15F1A9 (Charles Lindsey)
    news-standard,+content-md5/1,+content-type/1+content-md5/3,
    +content-type/3
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
MIME-Version: 1.0
Content-Type: multipart/news-groupinfo; boundary=88888888

This is a multipart message in MIME format.

--88888888
Content-Type: application news-groupinfo
Content-MD5: T7NtIdVqde62kheQuAHOaw==

For your newsgroups file:
comp.foo                For Foo discussions (Moderated)

--88888888
Content-Type: text/plain

comp.foo a moderated newsgroup which passed its vote for creation
by 424:8 as reported in news.announce.newgroups on 10 Feb 99.

--88888888
Content-Type: application/news-transmission
Content-MD5: +piSsoeNmdin5ukFQuFTlw==

Newsgroups: comp.foo
Path: not-for-relaying
Distribution: local
From: "Charles Lindsey" <[email-masked]>
Message-ID: <919190727.4918/[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Subject: Charter for newsgroup com.foo
Approved: [email-masked]

The charter, culled from the call for votes:

   Comp.foo is a moderated newsgroup for discussing all manner of
   Foos.

Moderation submission address:
   [email-masked]

--88888888--

5.2.  Mail message re-signed by mailing list owner

Resent-From: "Example Mail Server" <[email-masked]>
Verified: [email-masked]; hashcheck=content-md5
Signed-1: FOOBAR 0x2C15F1A9 message-id,date,resent-from,
        verified,signed
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
From: <"[john]"@
    temple.example> (John Smith)
Organization: http://www.temple.example/john
Subject: Submission to mailing list
      in connection with foo.
Date: Sat, 13 Feb 1999 22:59:46 GMT
Content-MD5: +piSsoeNmdin5ukFQuFTlw==
Signed: Foobar 0x2C15F1A9 (John Smith) mail-standard,content-md5
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8

Text of John's message.

--
John's signature.

   Passing this through the foobar canonicalization algorithm produces
   the following, in the case of the "Signed:" header (observe lines
   folded for convenience of this document - the true line endings
   indicated by "CRLF"):

signed:Foobar0x2C15F1A9mail-standard,content-md5CRLF
date:918946786CRLF
from:<"[john]"@temple.example>(John Smith)CRLF
subject: Submission to mailing list in connection with foo.CRLF
content-type:text/plain;charset=us-asciiCRLF
content-md5:+piSsoeNmdin5ukFQuFTlw==CRLF

   And here is the result of canonicalizing the "Signed-1:" header:

signed-1:FOOBAR0x2C15F1A9message-id,date,resent-from,verified,
signedCRLF
message-id:<[email-masked]>CRLF
date:918946786CRLF
resent-from:ExampleMailServer<[email-masked]>CRLF
verified:[email-masked];hashcheck=content-md5CRLF
signed:Foobar0x2C15F1A9mail-standard,content-md5iQCVAwUBNsm8pcJdOtO4ja
nBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501NqacLXEkQFi0I2sWh+9NSW4KdMqN
1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJ
up9lxreI7ASGuUQj6J1AorahY5xtw2ukbEJX/0slc==dTd8CRLF

        NOTE: the second signature signed only that which it had added
        itself, plus sufficient of the original headers to identify
        the original message. It did not need to scan the body to
        recompute the MD5 hash, but effectively included it by signing
        the original "Signed:" header.

6.  Security

   TBD
[What is there to say here?]

7.  References

   [RFC1036] M. Horton and R. Adams, "Standard for Interchange of
        USENET Messages", RFC 1036, December 1987.

   [RFC1327] S. Hardcastle-Kille, "Mapping between X.400(1988) / ISO
        10021 and RFC 822", RFC 1327, May 1992.

   [RFC1864] J. Myers and M. Rose, "The Content-MD5 Header Field", RFC
        1864, October 1995.

   [RFC2015] M. Elkins, "MIME Security with Pretty Good Privacy
        (PGP)", RFC 2015, October 1996.

   [RFC2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
        Extensions (MIME) Part One: Format of Internet Message
        Bodies", RFC 2045, November 1996.

   [RFC2047] K. Moore, "MIME (Multipurpose Internet Mail Extensions)
        Part Three: Message Header Extensions for Non-ASCII Text", RFC
        2047, November 1996.

   [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate
        Requirement Levels", RFC 2119, March 1997.

   [RFC2234] D. Crocker and P. Overell, "Augmented BNF for Syntax
        Specifications: ABNF", RFC 2234, November 1997.

   [RFC2440] J. Callas, L. Donnerhacke, H. Finney, and R. Thayer,
        "OpenPGP Message Format", RFC 2440, November 1998.

   [MESSFOR] P. Resnick, "Internet Message Format Standard", draft-
        ietf-drums-msg-fmt-04.txt, March 1998.

   [PGPMOOSE] Greg Rose, [I need a URL for this], October 1995.

   [PGPVERIFY] David Lawrence,
        ftp://ftp.isc.org/pub/pgpcontrol/README.html.

   [SMTP] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer
        Protocol", draft-ietf-drums-smtpupd-*.txt.

   [USEFOR] Dan Ritter, "News Article Format", draft-ietf-usefor-
        article-format-*.txt.

8.  Acknowledgements

   The author acknowledges the work of David Lawrence, as original
   author of "pgpverify", for many of the ideas contained herein, and
   also many contributions from members of the usenet-format mailing
   list.

9.  Author's address

   Charles Lindsey
   5 Clerewood Avenue, Heald Green, CHEADLE, SK8 3JU, U.K.
   <[email-masked]>

   This draft expires on
   TBD

Appendix A. Model implementation

[It is intended to include an implementation in PERL here.]

Appendix B. Test cases

[It is intended to include some pathological examples here for
implementations to be tested against.]

--7fbd_6533-127c_3cb6-5da4_1972--

Date: Tue, 16 Mar 1999 11:59:32 -0600
To: [email-masked], [email-masked]
From: Paul Hoffman / IMC <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

I don't see a need for this in the email world. In our many discussions of 
the topic, two themes have always emerged:
- Signing in nice but is rarely needed
- When it is needed, you can encapsulate the message with its headers and 
sign the result

--Paul Hoffman, Director
--Internet Mail Consortium

Date: Tue, 16 Mar 1999 17:45:33 +0000
To: [email-masked], [email-masked]
From: Paul Overell <[email-masked]>
Subject: Re: Draft for signed headers
Sender: [email-masked]

In article <[email-masked]>, Charles Lindsey
<[email-masked]> writes

Nit-picking on the canonicalization algorithm.

>   1.   The field-name (header-name) at the start of the header is
>        converted to lowercase.
>
>   2.   If the header is unstructured, all instances of FWS are
>        replaced by a single SPACE; otherwise (the header is
>        structured and) all instances of FWS are omitted, except
>        within comments where they are replaced by a single SPACE (the
>        header has now been unfolded into a single line). Any
>        whitespace at the end of the header is removed, and it is
>        ensured that the header ends with a single CRLF.
>
        In RFC2234 space is SP, not SPACE.

>   3.   All instances of DQUOTE (ASCII '"') are removed, except when
>        they occur between properly matched pairs of "<" and ">"
>        (thus, in particular, they are not removed within msg-ids).
>
                "<\">"@domain

        Are these < > properly matched?  Should this be canonicalized to

                <\">@domain

        or

                <\>@domain

        I can see what you are trying to do but I think that there may
        be several pitfalls here. 

>   4.   Any date-time occurring in a Date, Resent-Date or Expires
>        header (but not in any other header) is converted into the
>        number of seconds since the start of January 1st 1970 UTC,
>        expressed as a decimal number without leading zeroes.
>
        As phrased the number of seconds since the start of January 1st
        1970 UTC includes leap seconds.  But this will give software a
        problem: how can it be written to cope with messages in the
        future? as future leap-seconds are undecided.

        Better to exclude leap-seconds, in which case you might need a
        note about what to do with a seconds value of 60.

>   5.   Any sequence of octets of length not more than 75 and not
>        including any SPACE (and hence presumed present in the same
>        line prior to Step 2), and which satisfies the syntax for an
>        encoded-word [RFC2047], and which is not enclosed between
>        properly matched pairs of "<" and ">" is replaced by the
>        sequence of octets obtained by decoding it. This is done
>        irrespective of whether that encoded-word was syntactically
>        allowed to be present at that position in the header according
>        to [RFC2047] or any extension thereof.
>

                (\<)=?utf-8?q?=E2=82=A0?=(\>)

        Are these < > properly matched?  Should this be canonicalized to

                (\<)₠(\>)

        or not?

        (For those without utf-8 it's a euro symbol).

Regards

-- 
Paul Overell                                        T U R N P I K E  Ltd

Date: Tue, 16 Mar 1999 15:16:02 GMT
From: Charles Lindsey <[email-masked]>
To: [email-masked], [email-masked]
Subject: Draft for signed headers
Reply-To: [email-masked], [email-masked]
Cc: [email-masked]
Sender: [email-masked]

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Content-MD5: 5/sktzZqbCARhfqCUnLi/Q==
X-Sun-Data-Type: text

There is a strongly perceived need within the Usenet-Format group for
a reliable method for the digital signing of headers. It would, on the
face of it, be foolish to devise a mechanism applicable to news that
would not work also for email, even though that does indeed make the
mechanism more complex.

And even though there may not be such a strong requirement for this
facility in email, one can imagine some applications that would find it
useful.

The draft that I have prepared therefore tries to solve the problem in
both environments. However, before we get too bogged down in the details
of what I am proposing, I suggest we first establish the proper forum
for this discussion, and the form that it should take. For this, I need
some advice from the various IETF gurus on these lists.

1. In the first place, should I aim it at a Draft Standard, or at an
Experimental Protocol? Note that we shall eventually want to refer to it
in the final USEFOR document, which is certainly supposed to be on the
Standards Track, but it might still be useful (and quicker) to have an
Experimental Protocol before then.

2. For the moment, I am posting this to both the ietf-82 and the
usenet-format lists, with Reply-To set to both, and I request that the
debate continue in BOTH places for now. Would it be useful for me to set
up a mailing list somewhere that automatically posted to both?

3. In the slightly longer term (and especially if it is to become a
Draft Standard in due course) do I need to apply to the IETF to have a
proper working group set up? That seems perhaps a slight overkill, so
would formal discussion on these two lists in parallel be in order even
in the longer term?

4. And finally, please do not tear into the nitty gritty and the
details just yet. Let us discuss the overall principles first, and the
mechanisms. Clearly, there is much detail to be sorted out eventually,
but first things first.

Cc: to David Lawrence (who is on at least one of the lists) because it impinges 
upon his pgpverify.

Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

--7fbd_6533-127c_3cb6-5da4_1972
Content-Type: application/octet-stream; name=signed
Content-Transfer-Encoding: 7bit
Content-MD5: 4S3wGLHhncD0mqi41KTnCA==
Content-Description: signed
Content-Disposition: attachment; filename=signed

Network Working Group                        Charles H. Lindsey
Internet-Draft                               University of Manchester
                                             February 1999

                  Signed Headers in Mail and Netnews

Status of this Memo

   This document is an Internet-Draft. Internet-Drafts are working
   documents of the Internet Engineering Task Force (IETF), its
   areas, and its working groups. Note that other groups may also
   distribute working documents as Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six
   months and may be updated, replaced, or obsoleted by other
   documents at any time. It is inappropriate to use Internet-
   Drafts as reference material or to cite them other than as "work
   in progress."

   To view the entire list of current Internet-Drafts, please check
   the "1id-abstracts.txt" listing contained in the Internet-Drafts
   Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
   (Northern Europe), ftp.nis.garr.it (Southern Europe),
   munnari.oz.au (Pacific Rim), ftp.ietf.org (US East Coast), or
   ftp.isi.edu (US West Coast).

Abstract

   TBD

1.  Introduction

1.1.  Scope and Objectives

[This is a Draft of a Draft, for discussion within the USEFOR and
ietf-822 mailing lists until the best format for putting it forward
has been decided, in particular whether it should continue under one
or the other of those lists, or whether it needs a separate IETF
Working Group. It also needs to be decided whether it should be aimed
towards the Standards track, or at an Experimental Protocol.]

[Note that comments in square brackets like this one constitute meta-
text - not intended to form part of any final document.]

   "Netnews" is a set of protocols [USEFOR] that enables news
   "articles" to be broadcast to potentially-large audiences, using a
   flooding algorithm which propagates copies throughout a network of
   participating hosts. The huge growth in the use of this protocol in
   recent years has been accompanied by many attempts to abuse the
   system by causing it to appear that articles came from parties
   other than those that actually injected them, or that they had been
   posted with some Approval that the real poster was not entitled to
   give, or that they otherwise appeared to be different from what
   they actually were. The effects of such abuse are particularly
   accute in the case of "Control" articles which can cause newsgroups
   to be created or removed on hosts worldwide, or which can cause
   unauthorized deletion of articles already received and stored on
   such hosts. It is therefore considered essential to provide a
   cryptographically secure means whereby it can be established beyond
   doubt that the source and structure of articles are exactly as they
   purport to be.

   "Electronic Mail" is a system for routing "messages" [MESSFOR]
   between individual computer users, usually on a one-to-one basis.
   The formats of Mail messages and News articles have deliberately
   been made to be similar, so that messages may be gatewayed to news
   systems and vice-versa. Whilst the scope for an abuser to cause
   serious damage is smaller in the case of Mail as opposed to News
   (because of the absence of Control headers), it would still be an
   added safeguard to be able to secure Mail headers
   cryptographically, especially for mail circulated via a mailing
   list. Moreover, the possibility of gatewaying between news and mail
   make it essential that the same security techniques should work in
   both.

   An existing experimental protocol "pgpverify" [PGPVERIFY] is
   already in widespread use for authenticating Control messages for
   creating and removing newsgroups within Usenet, and has proven
   itself very successful in mitigating the effects of malicious
   attacks against the integrity of Usenet.  This present standard is
   largely based upon pgpverify; however, pgpverify is unsuitable for
   more widespread use as it stands because it is unable to cope with
   folded headers and with the changes a mail message is likely to
   undergo during transport. A second similar experimental protocol
   "pgpmoose" [PGPMOOSE] is also currently in use for protecting
   moderated newsgroups against unauthorized postings.

   There also exist protocols for the cryptographic signature of
   bodies of articles, notably S/Mime and PGP/Mime [RFC2015], and it
   is moreover common to sign such bodies using PGP alone without the
   use of Mime [RFC2045] et seq at all. However, these protocols
   cannot, by their nature, be used to sign headers. Moreover, since
   the signature is applied after any Content-Transfer-Encoding
   [RFC2045], it may be impossible to verify the signature if the
   Content-Transfer-Encoding should be changed as the message passes
   through a succession of sites during transport. Nevertheless, this
   present standard does not attempt to usurp those protocols, but
   merely provides the means to sign headers, both of complete
   messages and of headers embedded in Mime messages and multiparts.

[This document has been designed to fit on top of the drafts currently
in preparation for Mail [MESSFOR] and for News [USEFOR].  It is
expected that at least the Mail draft will have progressed to the RFC
stage by the time the present document is complete, at which time all
references to [MESSFOR] in the present text will be replaced by
references to that RFC. If it is thought wise to issue this document
before [USEFOR] is complete, then that reference will have to be to
[RFC1036] instead.]

1.2.  Notations and Conventions

1.2.1.  Requirements notation

   Certain words used to define the significance of individual
   requirements are capitalized. "MUST", "SHOULD", "MAY" and the same
   words followed by "NOT" should be read as having the same meaning
   as in [RFC2119].  In particular, to be fully compliant with this
   document, software must satisfy every relevant "MUST" requirement.
   Software that satisfies every relevant "SHOULD" requirement but not
   every "MUST" requirement is partially compliant.

1.2.2.  Syntactic notation

   This document uses the Augmented Backus Naur Form described in
   [RFC2234].  A discussion of this is outside the bounds of this
   document, but it is expected that implementors will be able to
   quickly understand it with reference to the defining document.

1.3.  Overview

   This standard makes provision for Signed headers to be included in
   mail and news articles and in Mime messages and multiparts. A
   Signed header provides a cryptographic signature over a named set
   of other headers, including lower level headers contained in Mime
   messages and multiparts below the current level. Such signatures
   can give assurance to a recipient who verifies them that those
   headers have not been changed or added to in transit, and/or that
   the article was indeed sent by its purported originator.

   The bodies of articles, Mime messages and multiparts are not
   directly included in the Signature. Rather, the intention is that
   each such body part should have a Content-MD5 (or similar) header
   computed for it, and that header should then be included in the
   Signature instead.

   There is also provision for Verified headers which may be added by
   agents that have checked a Signed header. Verified headers may
   themselves be included in further Signed headers; this may be
   especially useful in the case of gateways which find it necessary
   to change an article in ways that invalidate an original signature.

   Every effort has been made to ensure that signatures remain
   verifiable in spite of all reasonable (and even unreasonable)
   changes to which they may be subjected in transit. These include
   changes to the Content-Transfer-Encoding of body parts (a principle
   reason for including them only via the Content-MD5 header), changes
   in the order of headers and of their layout, and encodings and re-
   encodings of unusual character sets. This is to be achieved by
   converting headers into a canonical form before they are signed.
   New headers, yet to be invented, need provide no problem, and there
   is no commitment to any particular character set (provided header-
   names remain in ASCII, as at present).

   Provision is made for different protocols which may be required in
   the future.  However, this standard defines just one, recommended
   protocol, and it is not desirable that other protocols should be
   defined unless and until serious deficiencies in the existing ones
   have been revealed.

2.  Basic Structure of Authenticating Headers

   A Signed or a Verified header may appear in the headers of a mail
   message or a news article, or in the headers of a Mime multipart
   sub-part or of a Mime message/rfc822 object (or indeed of any
   similar Mime object yet to be invented). In all cases, the term
   "current level" encompasses the entire set of headers in that same
   object. Where the headers at the current level include a "Content-
   Type:  multipart/*" or "Content-Type:  message/*" header, lower-
   level headers can arise within its sub-parts.

2.1.  Syntax of the Signed header

      Signed         = "Signed" ["-" DIGIT] ":" SPACE
                       [CFWS] protocol CFWS key-id CFWS header-ref-list
                       CFWS signature CRLF
      protocol       = ietf-token / x-token
      key-id         = token
      header-ref-list= header-ref *( [CFWS] "," [CFWS] header-ref )
      header-ref     = [ "+" / "-" ] ( field-name *( "/" 1*DIGIT )
                       / "mail-standard" / "news-standard" )
      field-name     ; <see [MESSFOR]>
      CFWS           ; <see [MESSFOR]>
      FWS            ; <see [MESSFOR]>
      ietf-token     = <An extension token defined by a standards-track
                       or experimental protocol RFC and registered
                       with IANA>
      x-token        = The two characters "X-" or "x-" followed, with
                       no intervening white space, by any token>
      token          = 1*<any (US-ASCII) CHAR except SPACE, CTLs,
                       or tspecials> [CFWS]
      tspecials      = "(" / ")" / "<" / ">" / "@" /
                       "," / ";" / ":" / "\" / DQUOTE /
                       "/" / "[" / "]" / "?" / "="
      signature      = 1*( btext [FWS] )
      btext          = %x41-5A / %x61-7A / %x30-39 / "+" / "/" / "="
                       ; base 64 chars

        NOTE: A signature may end in FWS, but not in CFWS. The
        requirement for an explicit SPACE after the ":" is to ensure
        compatibility with the syntax of Netnews [USEFOR]; it is not
        strictly necessary for Mail.

   The use of a DIGIT in the Signed header allows for 10 distinct such
   headers at any one level. "Signed:" is synonymous with "Signed-0:".
   This is more than sufficient for the intended usage (it would be
   most unusual to get beyond Signed-2) whilst still permitting
   implementations to check header-names against a fixed list of valid
   names. There MUST NOT be more than one Signed header with the same
   DIGIT within one set of headers.

   Tokens used for key-ids are case-insensitive.  "Foobar" is the
   preferred protocol defined by this standard.  It is desirable to
   keep the number of recognized protocols to an absolute minimum, and
   it is anticipated that further protocols would only be needed in
   the event that serious cryptographic deficiencies were to be found
   in the existing ones.
[Obviously, "foobar" is just a placeholder for whatever name is
finally chosen.]

   The key-id identifies the key used to generate the signature in a
   notation dependent upon the protocol (but commonly "0x" followed by
   hexadecimal digits). The CFWS following it MAY include a comment
   containing an identification of the person or entity which created
   the signature.

   The header-ref-list indicates those header-refs, at or below the
   current level, which are covered by the signature. The ordering of
   this list is significant. A header-ref prefixed by a "+", or not
   prefixed at all, indicates a header-ref to be added to the list
   defined by those preceding it, and a header-ref prefixed by "-"
   indicates a header-ref to be removed from the header-refs defined
   by the list preceding it.

   The header-ref "mail-standard" is a macro representing a set of
   common mail headers that SHOULD normally be included when signing
   the headers of a mail message, and is defined as the list

      Date, From, Reply-To, To, Cc, In-Reply-To, References, Subject,
      Keywords, Content-Type, Content-ID

   The header-ref "news-standard" performs the same function for news
   articles, and is defined as the list

      Date, Newsgroups, Distribution, Message-ID, From, Reply-To,
      Followup-To, References, Subject, Keywords, Control, Content-Type,
      Content-ID
[Well, you can imagine those lists are going to be fiercely argued
over]

        NOTE: Those lists have carefully excluded those headers (such
        as Sender and Content-Transfer-Encoding) which are liable to
        be added or altered by sites downstream from the one which
        generated the Signed header. If some header-ref in the list
        matches no header in the actual article, then it comprises an
        assertion that no such header was present when the article was
        signed. Headers which are routinely added to or altered as the
        article progresses through transports (such as Path, Received
        and Xref) SHOULD NOT be included in a header-ref-list, and
        neither should any header which appears twice in the set of
        headers. A header-ref prefixed by "-" may be used to exclude
        any header-ref from one of the standard lists.

2.2.  Semantics of the Signed header

   Where the headers at the current level include a "Content-Type:
   multipart/*" or "Content-Type: message/*" header, lower-level
   headers within its sub-parts may be referenced as follows:

   (i)  A header-ref not postfixed by any "/ DIGIT"s references the
        header of that name, if any, at the current level. Header-refs
        are, for this purpose, considered as case-insensitive.

   (ii) A header-ref of the form "XXXX/<m>" (or "XXXX/<m>/<n>..."),
        where <m> and <n> are numbers and the current level contains a
        "Content-Type: multipart/*" header, references the header that
        would be referenced by "XXXX" alone (or by "XXXX/<n>...") in
        the <m>th sub-part of that multipart, that sub-part now being
        regarded as the current level.

   (iii)A header-ref of the form "XXXX/1", where the current level
        contains a "Content-Type: message/rfc822" header (or any other
        message type which provides for its own set of headers),
        references the header that would be referenced by "XXXX" alone
        in that message object.

   (iv) A header-ref that does not match up with multipart or message
        Content-Type headers as indicated above MUST NOT be used.

   (v)  For example "Content-MD5/3/2" references the Content-MD5
        header of the second part of a multipart, which is itself the
        third part of a multipart established at the current level.

   A protocol, as established by this standard or by any RFC which
   extends this standard, comprises two parts: a "canonicalization
   algorithm" and a "cryptographic algorithm".

   The signature of a Signed header is constructed in accordance with
   a given header-ref-list as follows:

   1.   A partial Signed header is constructed from the name of the
        protocol, the key-id identifying the cryptographic key used
        (possibly followed by a comment indicating the person or
        entity responsible), followed by the header-ref-list, its
        following CFWS and a CRLF.

   2.   The header-ref-list is reduced by expanding the macros "mail-
        standard" and/or "news-standard", removing from the preceding
        part of the list any header-ref prefixed by a "-", and
        removing any duplicates.

   3.   The partial Signed header followed by all the headers
        referenced by the reduced header-ref-list (being headers at
        the current level or encapsulated within multiparts at any
        lower level) are concatenated to produce a list of headers to
        be signed.

   4.   The list of headers to be signed is subjected to the
        canonicalization algorithm of the protocol to produce a
        canonicalized list.

   5.   The canonicalized list is subjected to the cryptographic
        algorithm of the protocol to produce an octet stream
        representing the signature.

   6.   If the octet stream as produced by the cryptographic algorithm
        is not already in the form of base64 characters, it is now
        encoded in base64 [RFC2045].  It is then split into convenient
        lines by the insertion of FWS, and appended to the partial
        Signed header after a FWS.

   7.   The Signed header thus constructed is then incorporated into
        the set of headers at the current level.

   The signature of a Signed header is verified as follows:

   1.   The signature is removed from the Signed header to give a
        partial Signed header.

   2-4. The corresponding steps of the process that constructed the
        header are taken, producing a canonicalized list.

   5.   The public key identified by the key-id is now used by the
        cryptographic algorithm of the protocol to verify the
        signature. This may result in a simple pass-fail, or it may
        return some indication of the privileges (such as the
        authority to issue certain news control messages or to manage
        some mailing list) enjoyed by the owner of that key.

   The purpose of a Signed header is solely to establish that the
   headers referenced in it were present in an article when that
   article passed through the hands of the person or entity that
   generated the signature (and hence that it did indeed pass through
   those hands). It SHOULD NOT be taken as an endorsement of whatever
   is contained in the body of the article. If the contents of the
   body require such endorsement, then the body SHOULD be signed
   separately, for example in accordance with PGP/Mime [RFC2015].
[Hmm! I expect that paragraph to raise some discussion.]

   Signatures will typically be generated by the originators of
   articles (to prove the origin), by moderators of moderated
   newsgroups (to testify to their Approved header), by managers of
   mailing lists, and by gateways. They SHOULD NOT be generated by
   intermediate transports and relayers through which the article
   might pass. This is intended to be an end-to-end protocol, and
   signatures SHOULD ONLY be added when new, hitherto unsigned,
   information is added. Moreover, the set of headers included within
   the signature SHOULD be no more than is necessary to achieve the
   security desired.

        NOTE: It will be observed that no provision has been made to
        include the bodies of an article or of its sub-parts in the
        signature. If (as will indeed often be the case) it is
        required to attest that the body (or sub-part) dispatched
        along with the set of headers is the same as the body that was
        delivered at the far end, then the proper procedure is to
        construct a Content-MD5 header [RFC1864] for that body (or
        sub-part) and to include that Content-MD5 amongst the headers
        that are signed. Doing it this way confers three advantages:
        a) The Content-MD5 header is constructed in such a way that it
        is immune to changes of Content-Transfer-Encoding to which an
        article, or its sub-parts, may be subjected during transport.
        b) Given that many user agents already routinely construct a
        Content-MD5 header, and verify it on receipt (a practice much
        to be commended), it should be possible to generate a Signed
        header without an extra pass through the entire body
        (especially in the common case where there are no sub-parts).
        This applies particularly in the case of additional signatures
        by moderators or mailing list managers, who may not need to
        examine the body at all.
        c) If a Content-MD5 header should fail to verify (perhaps
        because of some transmission error) the verification of a
        Signed header might still succeed, giving the recipient at
        least some partial information as to where any problem might
        lie.

        NOTE: If, at some future time, a Content-SHA1 header (or any
        similar header based upon a different hashing algorithm)
        should be invented, it could equally well be used for this
        purpose.

2.3.  Syntax of the Verified header

      Verified       = "Verified" ["-" DIGIT] ":" SPACE
                       [CFWS] [ "FAILED" CFWS ] name-addr
                       *( [CFWS] ";" [CFWS] parameter ) CRLF
      name-addr      ; <see [MESSFOR]>
      parameter      = ( "hashcheck" / "hashcheck-failed" )
                       [CFWS] "=" [CFWS]
                       ( header-ref / DQUOTE header-ref-list DQUOTE ) /
                       attribute "=" [CFWS] value
      attribute      = token
      value          = token / quoted-string
      quoted-string  ; <see [MESSFOR]>

   The use of a DIGIT in the Verified header allows for 10 distinct
   such headers in one article (or one set of headers at some lower
   level). "Verified:" is synonymous with "Verified-0:". Each Verified
   header MUST match some Signed header with the same DIGIT in that
   same set of headers. There MAY be more than one Verified header
   with the same DIGIT within one set of headers (but observe that it
   would not then be possible to include those headers in a further
   Signed header).

   Tokens used for attributes are case-insensitive. The only
   parameters defined by this standard are the "hashcheck" and
   "hashcheck-failed" parameters. Other parameters permitted by the
   syntax are for the purpose of future extensions to this standard,
   and should be ignored except as defined in such extensions.  The
   "value" part of a "hashcheck" parameter covering several header-
   refs is of the form of a quoted-string because in general it can
   contain "," characters (which are tspecials). Even a single
   header-ref MUST NOT be used without DQUOTEs if it contains any
   tspecial character (which is unlikely, but not forbidden by the
   syntax of [MESSFOR]).
[Do we also want a "confidence" parameter for the verifier to express
his certainty of the identity of the original Signer, and if so, what
notation to use?]

   The header-refs following any "hashcheck=" or "hashcheck-failed="
   MUST ONLY be of the form "Content-MD5", or of some other header
   based upon another hashing algorithm which may be invented in the
   future.

2.4.  Semantics of the Verified header

   The Verified header is intended to be added to an article by an
   agent through which the article passes, and serves as an assertion
   that the corresponding Signed header has been cryptographically
   verified by the person or entity identified in the name-addr (or
   otherwise if the optional "FAILED" is present). The addr-spec
   contained in that name-addr MUST be a valid email address by which
   that person or entity may be contacted. The original Signed header
   MUST NOT be removed from the article. The Verified header
   (supposing it is the only one present with that particular DIGIT)
   MAY itself be included in a further Signed header added at the same
   time.

        NOTE: The purpose of a Verified header is to save the ultimate
        recipient the trouble of verifying the cryptographic signature
        himself (which can be time consuming, and may require
        knowledge of public keys not in his possession).  Such a
        verification, if performed close to the ultimate recipient
        (such as by the news or mail server to which he connects)
        could normally be regarded as adequate evidence of
        authenticity, even if not signed itself. It would be hard
        (certainly in the case of news) for a malicious interloper to
        cause such a verification to appear bearing the identity of
        the local server of each ultimate recipient.

        NOTE: The Verified header is also useful in the case that a
        gateway (or a moderator) makes some change to an article that
        renders an original Signed header invalid. Such a gateway can
        therefore certify that the original form of the Signed header
        had been verified, and can then resign the article (including
        his added Verified header). Likewise, a site (such as the
        originator's own server) with a well known public key can
        verify and resign an article whose originator's public key may
        be less well known. However, Verified headers SHOULD NOT be
        added as routine by other intermediate sites.

   It is normally the business of the reading agent of the ultimate
   recipient to check the correctness of a Content-MD5 or similar
   header. Nevertheless, an earlier agent that has added a Verified
   header and also checked such a Content-MD5 header MAY so indicate
   by mentioning it in a "hashcheck" parameter (or "hashcheck-failed"
   as appropriate).

3.  Protocol definition

3.1.  Requirements for canonicalization algorithms

   It is a sad fact of life that those implementing agents for
   handling Netnews and Mail cannot resist the temptation to "improve"
   articles passed through them by rewriting headers that are thought
   not to conform to some real or supposed standard. Experience shows
   that, in the majority of cases, such tinkering makes matters worse
   rather than better, and for that reason [USEFOR] and, to a lesser
   extent, [MESSFOR] and [SMTP] try to forbid it, especially when
   perpetrated by relaying and transport agents (there are arguments
   in favour of allowing injecting agents and other agents close to
   the originator to do some limited cleanups, especially where it is
   impractical to return the article to the originator for
   correction).

   Furthermore, in the case of Mail it is often required for the
   transport protocols to modify articles en route, most notably when
   articles containing octets with the 8th bit set have to be passed
   through a channel that permits only 7bit.

   It is a further sad fact of life that agents which make such
   changes are not going to go away just because some standard says
   so. Therefore, the canonicalization algorithm SHOULD endeavour to
   enable the headers of articles to be signed and verified in
   accordance with this standard in spite of such tinkerings, insofar
   as they can be anticipated. The following list indicates some
   common practices which are worth detecting and protecting against.

   o    Headers may be re-folded to fit within some preferred overall
        line length.  This may result in the creation of whitespace
        where none existed before.

   o    Trailing whitespace may be removed, and line endings changed
        to/from CRLF.

   o    Field-names may be converted into some usual canonical form
        (e.g.  "Mime-Version" into "MIME-Version").

   o    Phrases, or parts thereof, may be converted to or from
        quoted-strings.

   o    Date-times may be rewritten in some preferred format, or into
        some preferred timezone.

   o    Headers with non-ASCII characters may be converted to or from
        the notation defined in [RFC2047].  Observe that there is no
        canonical way to do this conversion and it is, moreover,
        frequently performed in contexts where it is not strictly
        allowed.
[Other contributions to this list welcomed.]

   Since the slightest change to a canonicalization algorithm will
   render it inoperable with previous versions, such an algorithm MUST
   NOT be changed once it has been defined by this standard, or any
   extension thereof. In the event of some inadequacy being found, it
   would be necessary to devise and standardize a new algorithm, a
   task not to be undertaken lightly. For this reason,
   canonicalization algorithms SHOULD be designed to cope with the
   widest possible range of headers, including those not yet invented.
   Therefore, they SHOULD NOT, so far as possible, rely on the ability
   to parse any particular header.

        NOTE: A canonicalization algorithm is required simply to
        produce an octet stream for submission to the cryptographic
        algorithm. That stream does not have to be human readable, nor
        does it have to be a syntactically-correct header, nor does it
        have to be convertible back into the original header, or into
        any correct header at all. Insofar as many original headers
        can, in principle, be mapped into the same octet stream, this
        in no way reduces the utility of the algorithm, even though it
        might enable conspiracy theorists to imagine, and even
        implement, various sorts of covert channels for use by
        malicious interlopers.

3.2.  The Foobar protocol

[Suggestions for a proper name on a postcard, please, to /dev/null for
now.]

   The "foobar" protocol is comprised of a canonicalization algorithm
   "foo" and a cryptographic algorithm "bar".

3.2.1.  The Foo canonicalization algorithm

   For the purposes of this algorithm, the headers Subject, Comments,
   Organization and Summary, and all headers starting with "X-", are
   to be considered "unstructured" and all other headers "structured"
   (whether or not they were so described in any other standard). Each
   header to be included in the algorithm, which will in general
   consist of several lines (those after the first commencing with
   whitespace), is processed as follows:

   1.   The field-name (header-name) at the start of the header is
        converted to lowercase.

   2.   If the header is unstructured, all instances of FWS are
        replaced by a single SPACE; otherwise (the header is
        structured and) all instances of FWS are omitted, except
        within comments where they are replaced by a single SPACE (the
        header has now been unfolded into a single line). Any
        whitespace at the end of the header is removed, and it is
        ensured that the header ends with a single CRLF.

   3.   All instances of DQUOTE (ASCII '"') are removed, except when
        they occur between properly matched pairs of "<" and ">"
        (thus, in particular, they are not removed within msg-ids).

   4.   Any date-time occurring in a Date, Resent-Date or Expires
        header (but not in any other header) is converted into the
        number of seconds since the start of January 1st 1970 UTC,
        expressed as a decimal number without leading zeroes.

   5.   Any sequence of octets of length not more than 75 and not
        including any SPACE (and hence presumed present in the same
        line prior to Step 2), and which satisfies the syntax for an
        encoded-word [RFC2047], and which is not enclosed between
        properly matched pairs of "<" and ">" is replaced by the
        sequence of octets obtained by decoding it. This is done
        irrespective of whether that encoded-word was syntactically
        allowed to be present at that position in the header according
        to [RFC2047] or any extension thereof.

        NOTE: The order of these steps is important. The unfolding
        must occur before matched pairs of "<" and ">" can be
        detected. The decoding of supposed encoded-words must take
        place last, because it could produce arbitrary sequences of
        octets (when decoding into UCS-16, for example) which might
        then be confused with ASCII characters such as DQUOTE, etc.
        Whitespace needs to be removed entirely from structured
        headers because it is possible it may have been introduced by
        folding in unexpected places en route, subsequent to the
        original signing.

        NOTE: If an internet mail message in the format defined by
        [MESSFOR] is converted into X.400 mail by a gateway conforming
        to [RFC1327] and then back into internet mail, then it is
        likely that any signature made in accordance with this
        standard will fail to verify. For example comments in headers
        containing addresses (such as in From, Reply-To, etc.) may be
        converted into phrases and moved in front of the addr-spec, or
        even removed entirely, and thus the canonicalized form of the
        message will have been changed.  This old convention for
        storing the Real Name of the person associated with the
        address in a following comment, is now deprecated by both
        [MESSFOR] and [USEFOR], but even where phrases are used for
        this purpose it is possible that other changes to the message
        will still render the signature unverifiable. Note that there
        is in any case no expectation that an internet mail message
        signed according to this standard will ever be able to be
        verified once it has been passed permanently into an X.400
        system, nor vice versa.

[The problems of X.400 gateways gave me much concern. The only case of
interest is where an RFC822 message is bounced off an X.400 mailer (so
it gets converted both ways). This could happen without the parties at
either end being aware of the possibility, so it would be nice if it
worked. However, [RFC1327] is extremely hard to read, and I eventually
has recourse to experimentation with an actual X.400 gateway. Here are
examples of the sorts of mungings encountered, with "before" and
"after" versions of various headers:

From: [email-masked] (Charles H. Lindsey)
From: " (Charles H. Lindsey)" <...>

From: [email-masked] (Charles Lindsey) (Foo Bar)
From: " (Charles Lindsey) (Foo Bar)" <...>

Reply-To: Charles Lindsey <[email-masked]> (Foo Bar)
Reply-To: "Charles Lindsey (Foo Bar)" <...>

These examples show comments after addr-specs being turned to phrases
in front. Removing all comments during canonicalization would not help
here, which is the main reason for dropping that idea.

From: "Charles Lindsey" <[email-masked]>
From: Charles Lindsey <...>

Reply-To: Charles "H." Lindsey <[email-masked]>
Reply-To: "Charles H. Lindsey" <...>

These examples show reasonably intelligent handling of DQUOTE in
phrases, and justify their complete removal in the Foo algorithm.

From: [email-masked]
From: ...

Reply-To: <[email-masked]>
Reply-To: chl <...>

These were trying to establish whether it preserved presence or
absence of <...> around add-specs. It seems it does, but the added
"chl" is worrying.

To: "Autoanswer" <[email-masked]>
        (hopefully)
To: [email-masked]

To: [email-masked]
To: [email-masked]

It seems to throw away all Real Names in To: headers, and to lose the
<...> too.

Cc: [email-masked] (Charles Lindsey)
Cc: ...

And loses all Real Names in Cc:

Message-Id: <[email-masked]> (message)
Message-ID: <[email-masked]>

Comments in Message-IDs get lost. I do not think that matters too
much.

Date: Thu, 4 Mar 1999 19:40:18 GMT (UTC)
Date: Fri, 5 Mar 1999 03:12:54 +0000

It has changed the date when sent to the date when received at the
gateway!  This seems to me a violation of [RFC1327].

Mime-Version: 1.0 (MIME)
Mime-Version: 1.0 (MIME)

It leaves headers it does not need to understand completely alone.

My conclusion is that some messages passed through X.400 gateways are
never going to verify correctly, but that many straightforward ones
will survive with the Foo algorithm as proposed. I think that is the
best that can be done, and better than not doing anything at all.
There is certainly no benefit in removing comments entirely, as I had
once thought there might be.]

3.2.2.  The Bar cryptographic algorithm

[Open PGP is the obvious choice for this, since it is widely available
and is blessed by the IETF. My only reservation is that is comes with
a rather poor certification system as compared with, say, SPKI. So
this choice might yet have to be reviewed.]

   The stream of octets resulting from the canonicalization algorithm
   is signed, in binary mode (signature type 0x00), in accordance with
   Open PGP [RFC2440].

        NOTE: The signature is made in binary mode just in case any
        [RFC2047] decoding into UCS-16 has produced octets which might
        be mistaken for isolated CR, LF or trailing SPACE characters,
        which are treated specially in PGP text mode.

   The output of the algorithm MUST be Ascii-armored [RFC2440], but
   the Armor Header Line ("BEGIN PGP SIGNATURE"), the Armor Headers
   (e.g.  "Version:"), the blank line following the Armor Headers, and
   the Armor Tail ("END PGP SIGNATURE") are to be omitted (thus
   yielding a sequence of base64 characters). Observe that these
   characters will include a CRC checksum, which SHOULD be on a
   separate line from the rest of the signature.

   The signature included within the Ascii-armor MAY include
   certificates as evidence that the signing key has the necessary
   authorization to sign articles of that nature, but such usage is in
   general deprecated except between parties that have agreed
   otherwise or where, for some reason, an unusual signatory is
   signing and attaches a certificate from the usual signatory.

   The signature SHOULD use the DSA public-key algorithm and the SHA-1
   hashing algorithm, and be incorporated in a Version 4 Signature
   Packet in the new format. It MAY alternatively use the combination
   RSA/MD5 with Version 3 in the old format (for compatibility with
   PGP 2.6.x) and it MAY use the combination RSA/SHA-1 with Version 4
   in the new format. Verifiers MUST be able to verify all of these
   forms.

4.  Applications

   It is anticipated that protocols for specific applications of the
   signature mechanisms described in this standard will be devised,
   whether under the auspices of the IETF or otherwise. For example,
   the need to be able to verify the origin of Control messages for
   creating and removing newsgroups and for cancelling articles was a
   prime motivation for creating this standard.

   It is up to each such application to specify appropriate mechanisms
   for establishing a Public Key Infrastructure suited to its purpose.
   Such an infrastructure would provide for the storing, distribution
   and authorization of the necessary public keys (and for revocations
   thereof). This standard establishes no preferred mechanisms in this
   regard, except to draw attention to the possible usefulness of the
   Content-Type application/pgp-keys as defined in [RFC2015].

5.  Examples

[The MD5 hashes and signatures in the following are bogus, but I would
expect to include genuine verifiable ones in the final version.]

5.1.  Newgroup Control message

   A 'newgroup' control message in the format given in [USEFOR].

Newsgroups: comp.foo
From: "Charles Lindsey" <[email-masked]>
Subject: cmsg newgroup comp.foo moderated
Control: newgroup comp.foo moderated
Approved: [email-masked]
Message-ID: <[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Signed: foobar 0x2C15F1A9 (Charles Lindsey)
    news-standard,+content-md5/1,+content-type/1+content-md5/3,
    +content-type/3
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
MIME-Version: 1.0
Content-Type: multipart/news-groupinfo; boundary=88888888

This is a multipart message in MIME format.

--88888888
Content-Type: application news-groupinfo
Content-MD5: T7NtIdVqde62kheQuAHOaw==

For your newsgroups file:
comp.foo                For Foo discussions (Moderated)

--88888888
Content-Type: text/plain

comp.foo a moderated newsgroup which passed its vote for creation
by 424:8 as reported in news.announce.newgroups on 10 Feb 99.

--88888888
Content-Type: application/news-transmission
Content-MD5: +piSsoeNmdin5ukFQuFTlw==

Newsgroups: comp.foo
Path: not-for-relaying
Distribution: local
From: "Charles Lindsey" <[email-masked]>
Message-ID: <919190727.4918/[email-masked]>
Date: Tue, 16 Feb 1999 18:45:27 -0000
Subject: Charter for newsgroup com.foo
Approved: [email-masked]

The charter, culled from the call for votes:

   Comp.foo is a moderated newsgroup for discussing all manner of
   Foos.

Moderation submission address:
   [email-masked]

--88888888--

5.2.  Mail message re-signed by mailing list owner

Resent-From: "Example Mail Server" <[email-masked]>
Verified: [email-masked]; hashcheck=content-md5
Signed-1: FOOBAR 0x2C15F1A9 message-id,date,resent-from,
        verified,signed
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8
From: <"[john]"@
    temple.example> (John Smith)
Organization: http://www.temple.example/john
Subject: Submission to mailing list
      in connection with foo.
Date: Sat, 13 Feb 1999 22:59:46 GMT
Content-MD5: +piSsoeNmdin5ukFQuFTlw==
Signed: Foobar 0x2C15F1A9 (John Smith) mail-standard,content-md5
    iQCVAwUBNsm8pcJdOtO4janBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501
    NqacLXEkQFi0I2sWh+9NSW4KdMqN1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26
    VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJup9lxreI7ASGuUQj6J1AorahY5xtw2
    ukbEJX/0slc=
    =dTd8

Text of John's message.

--
John's signature.

   Passing this through the foobar canonicalization algorithm produces
   the following, in the case of the "Signed:" header (observe lines
   folded for convenience of this document - the true line endings
   indicated by "CRLF"):

signed:Foobar0x2C15F1A9mail-standard,content-md5CRLF
date:918946786CRLF
from:<"[john]"@temple.example>(John Smith)CRLF
subject: Submission to mailing list in connection with foo.CRLF
content-type:text/plain;charset=us-asciiCRLF
content-md5:+piSsoeNmdin5ukFQuFTlw==CRLF

   And here is the result of canonicalizing the "Signed-1:" header:

signed-1:FOOBAR0x2C15F1A9message-id,date,resent-from,verified,
signedCRLF
message-id:<[email-masked]>CRLF
date:918946786CRLF
resent-from:ExampleMailServer<[email-masked]>CRLF
verified:[email-masked];hashcheck=content-md5CRLF
signed:Foobar0x2C15F1A9mail-standard,content-md5iQCVAwUBNsm8pcJdOtO4ja
nBAQFo2gQAiF34beuKU3wwEpIv+yM+LdN72pW1f501NqacLXEkQFi0I2sWh+9NSW4KdMqN
1WgoiVlhWJU4KJYnvSJBigQHrlrLM1FWFw26VWNeICaOZUZAqwZdodVy2S0F1ilQ01lPTJ
up9lxreI7ASGuUQj6J1AorahY5xtw2ukbEJX/0slc==dTd8CRLF

        NOTE: the second signature signed only that which it had added
        itself, plus sufficient of the original headers to identify
        the original message. It did not need to scan the body to
        recompute the MD5 hash, but effectively included it by signing
        the original "Signed:" header.

6.  Security

   TBD
[What is there to say here?]

7.  References

   [RFC1036] M. Horton and R. Adams, "Standard for Interchange of
        USENET Messages", RFC 1036, December 1987.

   [RFC1327] S. Hardcastle-Kille, "Mapping between X.400(1988) / ISO
        10021 and RFC 822", RFC 1327, May 1992.

   [RFC1864] J. Myers and M. Rose, "The Content-MD5 Header Field", RFC
        1864, October 1995.

   [RFC2015] M. Elkins, "MIME Security with Pretty Good Privacy
        (PGP)", RFC 2015, October 1996.

   [RFC2045] N. Freed and N. Borenstein, "Multipurpose Internet Mail
        Extensions (MIME) Part One: Format of Internet Message
        Bodies", RFC 2045, November 1996.

   [RFC2047] K. Moore, "MIME (Multipurpose Internet Mail Extensions)
        Part Three: Message Header Extensions for Non-ASCII Text", RFC
        2047, November 1996.

   [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate
        Requirement Levels", RFC 2119, March 1997.

   [RFC2234] D. Crocker and P. Overell, "Augmented BNF for Syntax
        Specifications: ABNF", RFC 2234, November 1997.

   [RFC2440] J. Callas, L. Donnerhacke, H. Finney, and R. Thayer,
        "OpenPGP Message Format", RFC 2440, November 1998.

   [MESSFOR] P. Resnick, "Internet Message Format Standard", draft-
        ietf-drums-msg-fmt-04.txt, March 1998.

   [PGPMOOSE] Greg Rose, [I need a URL for this], October 1995.

   [PGPVERIFY] David Lawrence,
        ftp://ftp.isc.org/pub/pgpcontrol/README.html.

   [SMTP] John C. Klensin and Dawn P. Mann, "Simple Mail Transfer
        Protocol", draft-ietf-drums-smtpupd-*.txt.

   [USEFOR] Dan Ritter, "News Article Format", draft-ietf-usefor-
        article-format-*.txt.

8.  Acknowledgements

   The author acknowledges the work of David Lawrence, as original
   author of "pgpverify", for many of the ideas contained herein, and
   also many contributions from members of the usenet-format mailing
   list.

9.  Author's address

   Charles Lindsey
   5 Clerewood Avenue, Heald Green, CHEADLE, SK8 3JU, U.K.
   <[email-masked]>

   This draft expires on
   TBD

Appendix A. Model implementation

[It is intended to include an implementation in PERL here.]

Appendix B. Test cases

[It is intended to include some pathological examples here for
implementations to be tested against.]

--7fbd_6533-127c_3cb6-5da4_1972--

Date: Tue, 16 Mar 1999 16:44:41 +0100 (MET)
From: Jean-Francois Stenuit <[email-masked]>
To: [email-masked], [email-masked]
cc: [email-masked]
Subject: Re: Draft for signed headers
Sender: [email-masked]

On Tue, 16 Mar 1999, Charles Lindsey wrote:

> There is a strongly perceived need within the Usenet-Format group for
> a reliable method for the digital signing of headers. It would, on the
> face of it, be foolish to devise a mechanism applicable to news that
> would not work also for email, even though that does indeed make the
> mechanism more complex.

IMHO, there should be two different schemes for NNTP and SMTP messages,
because the reason for signing the header is different.

For instance, we must be able to put multiple signatures on a usenet
message (one from the author, another from the administrator of the server
used for posting the article, and another for the moderator) because this
signing will be used mainly for cancelling an article. As one does not
usually cancel a mail message, this is not required for mail.

For mail applications, I don't see the point in signing the headers, as
long as the body is signed. But I may have missed a point, since I don't
follow the ietf-822 discussions.

> And even though there may not be such a strong requirement for this
> facility in email, one can imagine some applications that would find it
> useful.
> 
> The draft that I have prepared therefore tries to solve the problem in
> both environments. However, before we get too bogged down in the details
> of what I am proposing, I suggest we first establish the proper forum
> for this discussion, and the form that it should take. For this, I need
> some advice from the various IETF gurus on these lists.
> 
> 1. In the first place, should I aim it at a Draft Standard, or at an
> Experimental Protocol? Note that we shall eventually want to refer to it
> in the final USEFOR document, which is certainly supposed to be on the
> Standards Track, but it might still be useful (and quicker) to have an
> Experimental Protocol before then.

Here is also a difference, the need is urgent for usenet (because of the
ongoing cancel wars and other unauthenticated spam) but less urgent for
mail, as mail servers around the globe are becoming less prone to spam
propagation.

> 3. In the slightly longer term (and especially if it is to become a
> Draft Standard in due course) do I need to apply to the IETF to have a
> proper working group set up? That seems perhaps a slight overkill, so
> would formal discussion on these two lists in parallel be in order even
> in the longer term?

I find that having a single mailing list for "message format" is a good
idea.

> 4. And finally, please do not tear into the nitty gritty and the
> details just yet. Let us discuss the overall principles first, and the
> mechanisms. Clearly, there is much detail to be sorted out eventually,
> but first things first.

agreed.

Regards,

--
 Opinions expressed above are mine, they have not to be taken as a
 skynet official position.
 >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
/Jean-Francois "Jef" Stenuit | BELGACOM-Skynet NV/SA | Solaris 2.x,  \
\Network operations manager  | 124, Rue Col. Bourg   | Linux, NT,    /
/Phone (32)(2) 706-1311      | B-1140 Brussels       | Cisco         \
\Fax   (32)(2) 706-1312      | Belgium               | expert ...    /
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

To: [email-masked]
Xref: clerew local.mime:140
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Mon, 15 Mar 1999 14:35:33 GMT
Lines: 41
Sender: [email-masked]

In <[email-masked]> Randall Gellens <[email-masked]> writes:

>At 11:00 AM +0000 3/11/99, Charles Lindsey wrote:

>> No, I still think it is too slim. Consider the range of possibilities that
>> might be provided:

>[snip]

>I still think the proposal is best kept as is, at least for the 
>initial version.  I really don't want to add any more complexity.  We 
>already have enough (maybe too much).

I don't like the idea of an 'initial version' - it implies that a more
complex version will follow later. Trouble is, people who implement the
'initial version' will be none too happy if a 'revised version' follows
too soon afterwards.

But in fact, what I suggest is not significantly more complex than what is
proposed already (which is itself an 'uncomplex' as anything I have seen).

It is already the case that the implementer has to keep a count of '>' at
the beginning of a line, and then to do the line wrapping without those
characters, and finally but them back after the wrapping.

What I am proposing merely requires a count of the SPACE characters at the
beginning of the line (with some allowance for TABs), to be counted AFTER
the '>' have been removed, but then treated in precisely the same way
(i.e. count them, remove them, wrap the line, and put them back).

If you want the 'exdent' feature as well, then you need to look also at
the space count of the following line.

That is not complex, and it would improve the usefulness of the feature
immeasurably.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:141
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: IETF Draft: Good Mailing List Behaviour
Date: Mon, 15 Mar 1999 14:41:27 GMT
Lines: 35
Sender: [email-masked]

In <[email-masked]> Pete Resnick <[email-masked]> writes:

>On 3/12/99 at 12:16 PM +0000, Charles Lindsey wrote:

>There is no current Resent-By field, though we could invent one. However:

Yes, I meant "Resent-From", didn't I :-( .

>> People keep saying that the Resent headers are not intended for 
>> mailing lists, but if not, what _are_ they intended for? And how am 
>> I supposed to deduce from the text of DRUMS why they are not 
>> suitable for this purpose?

>To quote DRUMS, Resent fields are intended to note that a message was 
>"reintroduced into the transport system by a user". A user. Not a 
>process. 

But the use of automatic mailing systems of all sorts has got us all used
to the idea that "users" include bots of all sorts. What other provisions
of DRUMS will suddenly become ineffective if you decide that the whole
thing only applies to human users?

But looking at it again, would it not be better to admit that these
headers ARE useful for mailing lists. Surely SOMETHING is needed so that
you can tell your message had come via a mailing list, and it is also
needed for loop detection (Received headers are too messy for that). The
List-* headers do not seem quite right for the job (they are, in any case,
intended for a slightly different purpose - namely facilitiating
(un)subscription). So why not make Resent "official"?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

From: Keith Moore <[email-masked]>
To: [email-masked] (Charles Lindsey)
cc: [email-masked]
Subject: Re: IETF Draft: Good Mailing List Behaviour 
Date: Fri, 12 Mar 1999 15:40:16 -0500
Sender: [email-masked]

>  People keep saying that the Resent headers are
> not intended for mailing lists, but if not, what _are_ they intended for?

manual resending, where you want the "resent" recipient to get the 
same headers.  it isn't used very often, but it's extremely valuable
for the cases where you do need it - e.g. to reroute a misdirected
message to the correct recipient, especially to a mail robot.

> And how am I supposed to deduce from the text of DRUMS why they are not
>  suitable for this purpose?

if it's not clear, we need to fix it.

Keith

Date: Fri, 12 Mar 1999 12:34:13 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 11:00 AM +0000 3/11/99, Charles Lindsey wrote:

>>The original version of the draft permitted either ">" or "> "; a
>>subsequent version used "> " only, and now we are at ">" only.
>
> Actually, you could allow either by letting the dequote/destuff cycle
> iterate until exhaustion.

I'd rather keep things as two separate steps: first dequote, then 
unstuff.  Otherwise you can't use space-stuffing to send a line with 
">" as the first character, which is kind of the point.  It also 
keeps things simpler.

>>Since this is a new format, I think there are several advantages to
>>picking one canonical quote indicate and sticking to it.  Allowing
>>options, or making the quote indicator an additional C-T parameter,
>>adds complexity with very little benefit.
> Indeed. The first quote character is under the control of the person who
> generates the flowed format, so you _could_ insist on ">" as the only
> supported possibility. However, the real problem is texts that come with a
> variety of quoting standards, having lines beginning with ">|:" for 3
> depths of quotation. Trouble is that many mailers allow you to choose your
> quote character, and some people cannot resist the tmeptation to take full
> advantage of all bells and whistles provided :-( . OTOH, I see no obvious
> solution to that problem.

Previously-quoted text is either flowed or fixed.  If it is flowed, 
we have no problem.  If it is fixed, the safest action is to keep it 
fixed.  If a mailer wants to convert from fixed to flowed, it will 
need to guess at quote characters and other aspects, probably with 
user input or direction.  Either way, we are OK.  If the text is kept 
as fixed, we are not using flowed, and so are no worse off then we 
are today.  If the text is converted to flowed, it will obey the 
flowed notation.

>>Yes, it is a slim proposal, that solves mainly one problem.  That is
>>the goal.  I think it is very important to have something that is
>>very simple and can be quickly implemented.  Anything more complex
>>will be harder to get consensus on, and take longer to get it right
>>and then implemented and deployed.  So, I take your comment as
>>evidence that this is on the right track :-).
>
> No, I still think it is too slim. Consider the range of possibilities that
> might be provided:

[snip]

I still think the proposal is best kept as is, at least for the 
initial version.  I really don't want to add any more complexity.  We 
already have enough (maybe too much).

Date: Fri, 12 Mar 1999 09:57:25 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
To: [email-masked]
Cc: [email-masked]
Sender: [email-masked]

> >It should also be mentioned what happens if a sender does space-stuffing
> >and the receiver doesn't know to undo space-stuffing - which would be
> >typical for all MUAs that exist today which don't understand
> >Format=Flowed.  Such text might best belong in the Failure Modes section
> >(which might be renamed "Interoperability Issues" perhaps).  I don't have
> >any text to recommend for this, though.

> I am in any case curious as to why space was chosen for the stuffing
> character. I would have expected `\`, which is more usual in such
> situations, and would make it immediately apparent to a user using an
> unmodified reader that some special convention was being used.

The answer is quite simple: You can see \, whereas space is invisible.

If we learned nothing else from quoted-printable, it is that people will
not deploy mechanisms that make their text look ugly. And \ definitely
qualifies.

				Ned

Date: Fri, 12 Mar 1999 11:30:31 -0600
To: [email-masked] (Charles Lindsey)
From: Pete Resnick <[email-masked]>
Subject: Re: IETF Draft: Good Mailing List Behaviour
Cc: [email-masked]
Sender: [email-masked]

On 3/12/99 at 12:16 PM +0000, Charles Lindsey wrote:

> I am still waiting for someone to explain to me why Resent-By: is 
> not suitable for this purpose.

There is no current Resent-By field, though we could invent one. However:

> People keep saying that the Resent headers are not intended for 
> mailing lists, but if not, what _are_ they intended for? And how am 
> I supposed to deduce from the text of DRUMS why they are not 
> suitable for this purpose?

To quote DRUMS, Resent fields are intended to note that a message was 
"reintroduced into the transport system by a user". A user. Not a 
process. Not a redirecter of some sort. They mean "Here is 
information about some person who resent this mail to you even though 
the rest of the fields don't indicate that it was coming from him to 
you." They are simply trace information to figure out why you got 
this message, where it came from, when it was sent to you, etc. The 
original From/Sender/To/Cc/Date/Message-ID fields are are intact so 
you can see the original information about the message.

pr
--
Pete Resnick <mailto:[email-masked]>
Eudora Engineering - QUALCOMM Incorporated
Ph: (217)337-6377 or (619)651-4478, Fax: (619)651-1102

To: [email-masked]
Xref: clerew local.mime:135
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: IETF Draft: Good Mailing List Behaviour
Date: Fri, 12 Mar 1999 12:16:24 GMT
Lines: 21
Sender: [email-masked]

In <[email-masked]> "D. J. Bernstein" <[email-masked]> writes:

>
>> (1) Add a "Received:" header to all messages passing the list. If a
>>     mailing list recognizes its own "Received:" header in an incoming
>>     message, such a message is dropped.

>In practice, people use fields that are easier to parse than Received,
>notably X-Loop and Delivered-To.

I am still waiting for someone to explain to me why Resent-By: is not
suitable for this purpose. People keep saying that the Resent headers are
not intended for mailing lists, but if not, what _are_ they intended for?
And how am I supposed to deduce from the text of DRUMS why they are not
suitable for this purpose?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:134
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Fri, 12 Mar 1999 12:13:40 GMT
Lines: 22
Sender: [email-masked]

In <[email-masked]> Dan Wing <[email-masked]> writes:

>  Space-stuffing is done by the sender and undone by the receiver, thus
>  causes no formatting changes to the message body.

>It should also be mentioned what happens if a sender does space-stuffing
>and the receiver doesn't know to undo space-stuffing - which would be
>typical for all MUAs that exist today which don't understand
>Format=Flowed.  Such text might best belong in the Failure Modes section
>(which might be renamed "Interoperability Issues" perhaps).  I don't have
>any text to recommend for this, though.

I am in any case curious as to why space was chosen for the stuffing
character. I would have expected `\`, which is more usual in such
situations, and would make it immediately apparent to a user using an
unmodified reader that some special convention was being used.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: [email-masked]
Xref: clerew local.mime:133
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Fri, 12 Mar 1999 12:11:01 GMT
Lines: 18
Sender: [email-masked]

In <[email-masked]> [email-masked] writes:

>Is the proletext proposal something that would be upward-compatible
>with the current F=F draft?  Or can it be made so?  The last thing
>we need to do is progress the current draft, and then go look
>at a subtly different scheme from proletext - you *know* that at
>least one major vendor will attempt to re-use the display engine
>and Get It Very Wrong if it's incompatible. ;)

I think so. But I doubt proletext would ever be just another format of
text/plain if/when it got proposed for an RFC. It would presumably be
text/proletext.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

Date: Thu, 11 Mar 1999 18:59:39 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: RE: Funny disruption of subject lines
To: Pete Resnick <[email-masked]>
Cc: Ned Freed <[email-masked]>, Dan Wing <[email-masked]>, "Larry Osterman (Exchange)" <[email-masked]>, "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Sender: [email-masked]

> > The general rule is that wherever this standard allows for folding
> > white space (not simply WSP characters), a CRLF may be inserted
> > before any WSP.

> In this sense, the CRLF is the *only* thing that is "inserted" when
> you are folding, and is the only thing that is "removed" when you are
> unfolding.

And that's how we're implementing it in light of 822bis. But while this may be
how we want it to be, it is far from being how things currently work.
Applications which depend on preservation of whitespace in headers don't
work very well in the real world.

				Ned

Date: Thu, 11 Mar 1999 19:12:07 -0600
To: Ned Freed <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: RE: Funny disruption of subject lines
Cc: Dan Wing <[email-masked]>, "Larry Osterman (Exchange)" <[email-masked]>, "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Sender: [email-masked]

On 3/11/99 at 4:35 PM -0800, Ned Freed wrote:

> It depends on the header field. Subject lines and other free-form fields
> used to only compress at wrapping points, and now don't compress at all.
>
> When structured fields are rewritten, however, the entire field ends up
> getting compressed.

This I think is reasonable. The elements in structured field each 
have specific semantics, so getting rid of spaces or comments between 
them to get at those elements is non-problematic. For things that are 
unstructured like Subject, losing whitespace may lose real semantics.

> At 4:24 PM -0800 3/11/99, Dan Wing wrote:
>
>> On Thu, 11 Mar 1999 17:36 -0600, Pete Resnick wrote:
>>
>>> The rule (which is clear in the DRUMS rewrite of 822) is that you 
>>> remove only the CRLF. You do not do any "space reduction".
>>
>> Agreed for unfolding.
>>
>> But for folding, doesn't this ABNF:
>>
>>>    FWS             =       ([*WSP CRLF] 1*WSP) /   ; Folding white space
>>>                            obs-FWS
>>
>> allow me to place an arbitrary number of spaces (or tabs) on the 
>> folded line?
>
> I believe this is correct.

FWS is *not* the "folding rule". FWS is a place in a production where 
white space may occur that can have folding in it. The folding rule 
is stated in 2.2.3:

> The general rule is that wherever this standard allows for folding 
> white space (not simply WSP characters), a CRLF may be inserted 
> before any WSP.

In this sense, the CRLF is the *only* thing that is "inserted" when 
you are folding, and is the only thing that is "removed" when you are 
unfolding.

pr
--
Pete Resnick <mailto:[email-masked]>
Eudora Engineering - QUALCOMM Incorporated
Ph: (217)337-6377 or (619)651-4478, Fax: (619)651-1102

Date: Thu, 11 Mar 1999 16:35:52 -0800 (PST)
From: Ned Freed <[email-masked]>
Subject: RE: Funny disruption of subject lines
To: Dan Wing <[email-masked]>
Cc: Pete Resnick <[email-masked]>, "Larry Osterman (Exchange)" <[email-masked]>, "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Sender: [email-masked]

> On Thu, 11 Mar 1999 17:36 -0600, Pete Resnick wrote:

> > On 3/11/99 at 3:19 PM -0800, Dan Wing wrote:
> >
> > > But this isn't the correct way to unwrap?  Unwrapping should yield one
> > > space between characters, not multiple spaces.
> >
> > Says who?

> I'll cite implementations, but I doubt that is what you were really
> asking.

> Netscape V4.5, PMDF's UA (V5.1-7), MultiNet's SMTP, and MX (a VMS mailer)
> have the space reduction behavior to varying degrees.

Actually, PMDF used to, but the current version (V5.2-31) doesn't any
more. We changed it to match DRUMS. (And we've had some interoperability
problems as a result, but that's another story...)

> Netscape compresses
> everything to one space, PMDF seems to compress only the wrapped portions,

It depends on the header field. Subject lines and other free-form fields
used to only compress at wrapping points, and now don't compress at all.

When structured fields are rewritten, however, the entire field ends up
getting compressed.

> MultiNet and (I believe) MX only compresses the wrapped portions.

> > The rule (which is clear in the DRUMS rewrite of 822) is that you
> > remove only the CRLF. You do not do any "space reduction".

> Agreed for unfolding.

> But for folding, doesn't this ABNF:

>   FWS             =       ([*WSP CRLF] 1*WSP) /   ; Folding white space
>                           obs-FWS

> allow me to place an arbitrary number of spaces (or tabs) on the folded
> line?

I believe this is correct.

				Ned

Date: Thu, 11 Mar 1999 16:24:22 -0800 (PST)
From: Dan Wing <[email-masked]>
To: Pete Resnick <[email-masked]>
cc: "Larry Osterman (Exchange)" <[email-masked]>, "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Subject: RE: Funny disruption of subject lines
Sender: [email-masked]

On Thu, 11 Mar 1999 17:36 -0600, Pete Resnick wrote:

> On 3/11/99 at 3:19 PM -0800, Dan Wing wrote:
> 
> > But this isn't the correct way to unwrap?  Unwrapping should yield one
> > space between characters, not multiple spaces.
> 
> Says who?

I'll cite implementations, but I doubt that is what you were really
asking.

Netscape V4.5, PMDF's UA (V5.1-7), MultiNet's SMTP, and MX (a VMS mailer)  
have the space reduction behavior to varying degrees.  Netscape compresses
everything to one space, PMDF seems to compress only the wrapped portions,
MultiNet and (I believe) MX only compresses the wrapped portions.

> The rule (which is clear in the DRUMS rewrite of 822) is that you
> remove only the CRLF. You do not do any "space reduction".

Agreed for unfolding.

But for folding, doesn't this ABNF:

  FWS             =       ([*WSP CRLF] 1*WSP) /   ; Folding white space
                          obs-FWS

allow me to place an arbitrary number of spaces (or tabs) on the folded
line?

-Dan Wing

Date: 12 Mar 1999 00:20:18 -0000
Mail-Followup-To: [email-masked]
From: "D. J. Bernstein" <[email-masked]>
To: [email-masked]
Subject: Re: IETF Draft: Good Mailing List Behaviour
Sender: [email-masked]

Jacob Palme writes:
> - Members only:     Only members are allowed to submit to the list

This causes huge problems with sublists, newsgroups, multiple accounts,
crossposted discussions, and other situations where a reader isn't
necessarily a subscriber.

> (1) Add a "Received:" header to all messages passing the list. If a
>     mailing list recognizes its own "Received:" header in an incoming
>     message, such a message is dropped.

In practice, people use fields that are easier to parse than Received,
notably X-Loop and Delivered-To.

> No non-delivery notification
>     should be sent in this case (since it might cause another loop).

Hiding loops is a horrible idea. Bouncing will never cause a loop if
it's done correctly; see http://pobox.com/~djb/proto/mailloops.txt for
details.

> (2) Store a data base of the Message-ID-s of messages which have
>     passed the list, and reject incoming messages whose Message-ID
>     is on this list.

As noted in http://pobox.com/~djb/proto/immhf/thread.html, such uses of
Message-ID are neither secure nor reliable.

> Apart from what is specifed in sections 8. Loop Control and 9. List
> Headers, a mailing list expander should not in any way modify the
> heading of a message.

This is astoundingly shortsighted.

---Dan

Date: Thu, 11 Mar 1999 15:36:04 -0800 (PST)
From: Dan Wing <[email-masked]>
To: [email-masked]
cc: [email-masked]
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

Section 5.4, "Space Stuffing", I would add some words indicating that the
space stuffing is similar to dot-stuffing already present in MTAs, perhaps
a sentence like:

  Space-stuffing is done by the sender and undone by the receiver, thus
  causes no formatting changes to the message body.

It should also be mentioned what happens if a sender does space-stuffing
and the receiver doesn't know to undo space-stuffing - which would be
typical for all MUAs that exist today which don't understand
Format=Flowed.  Such text might best belong in the Failure Modes section
(which might be renamed "Interoperability Issues" perhaps).  I don't have
any text to recommend for this, though.

-Dan Wing

Date: Thu, 11 Mar 1999 17:36:29 -0600
To: Dan Wing <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: RE: Funny disruption of subject lines
Cc: "Larry Osterman (Exchange)" <[email-masked]>, "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Sender: [email-masked]

On 3/11/99 at 3:19 PM -0800, Dan Wing wrote:

> But this isn't the correct way to unwrap?  Unwrapping should yield one
> space between characters, not multiple spaces.

Says who? The rule (which is clear in the DRUMS rewrite of 822) is 
that you remove only the CRLF. You do not do any "space reduction".

pr
--
Pete Resnick <mailto:[email-masked]>
Eudora Engineering - QUALCOMM Incorporated
Ph: (217)337-6377 or (619)651-4478, Fax: (619)651-1102

Date: Thu, 11 Mar 1999 15:19:40 -0800 (PST)
From: Dan Wing <[email-masked]>
To: "Larry Osterman (Exchange)" <[email-masked]>
cc: "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Subject: RE: Funny disruption of subject lines
Sender: [email-masked]

On Wed, 3 Mar 1999 10:22 -0800, Larry Osterman (Exchange) wrote:

> Subject: This is a message with an indescribably long subject.  I don't know
> why anyone would ever want to have a subject that is this long, but there
> are people who will want this.
> 
> Subject: This is a message with an indescribably long subject.  I 
>         don't know why anyone would ever want to have a subject that 
>         is this long, but there are people who will want this.
> 
> I believe that Exchange puts spaces in when it folds the lines, but it's
> entirely possible that some client is changing the spaces into tabs (it's
> all LWSP, right?).  What I cannot explain is the level of degradation that
> your subject is showing, but it's possible you are seeing a combination of
> these two effects (the "Re:" expansion and the long subject line expansion.

Improper unfolding causes the problem.  

Here is why someone might be tempted to unfold incorrectly. 

Let's say you're trying to send a subject line like:

  Subject: buying  a car  in  the countryside
                 ^^     ^^  ^^

where each of the areas marked with "^" are two spaces.  Pretend it
is a really long header and something along the way decides to
wrap it like this (obviously nothing would wrap it this tightly, but
I'm using a simple example because of the line-length limitations of
email).

  Subject: buying  a car
           in
           the
           countryside

when it gets unwrapped it will be unwrapped as:

  Subject: buying  a car in the countryside

which isn't what the sender sent!

An implementor trying to preserve the original spacing would wrap it
like this:

  Subject: buying  a car
    in
    the
   countryside

which would be unwrapped by that implementor's software like this:

  Subject: buying  a car  in  the countryside

which is exactly what the user sent.  Happy implementor!

But this isn't the correct way to unwrap?  Unwrapping should yield one
space between characters, not multiple spaces.

There isn't a way to prserve multiple spaces in a wrapped subject line
when the wrapping occurs at one of the multiple spaces.

-Dan Wing

From: Keith Moore <[email-masked]>
To: Jamie Zawinski <[email-masked]>
cc: Keith Moore <[email-masked]>, [email-masked]
Subject: Re: multipart + quoted-printable question 
Date: Thu, 11 Mar 1999 16:26:58 -0500
Sender: [email-masked]

> Why do you think it's ambiguous?

the text describing quoted-printable is written in an odd way.
it talks about how you encode long lines using soft line breaks, 
but doesn't talk about how you decode lines ending in =.

Keith

Date: Thu, 11 Mar 1999 13:13:12 -0800
From: Jamie Zawinski <[email-masked]>
Organization: mozilla dot org
To: Keith Moore <[email-masked]>
CC: [email-masked]
Subject: Re: multipart + quoted-printable question
Sender: [email-masked]

Keith Moore wrote:
> 
> I've come across what looks like an ambiguity in the MIME specs.

Why do you think it's ambiguous?

> Consider the following bodypart within a multipart with boundary "foo":
> 
> --foo
> Content-transfer-encoding: quoted-printable
> 
> =01=02=03
> --foo
> 
> what is the canonical form of the body part ?
> 
> is it: (hex) 01 02 03
> or: (hex) 01 02 03 0D 0A ?
> 
> (I think it's the former)
> 
> in other words, does the fact that the initial CRLF is attached to
> the boundary marker and not part of the preceding body part, mean
> that the preceding body part (in this case) doesn't end with a CRLF?

Yes, I think it doesn't end with CRLF.

> Now consider this body part (ending with a soft line break):
> 
> --foo
> Content-transfer-encoding: quoted-printable
> 
> =01=02=03=
> --foo
> 
> is the canonical form of this body part also (hex) 01 02 03?
> or is this just illegal because there's no following CRLF?

I think that's illegal just as

=01=02=0
--foo

would be illegal.

-- 
Jamie Zawinski
[email-masked]    http://www.mozilla.org/   (work)
[email-masked]        http://www.jwz.org/       (play)

Date: Thu, 11 Mar 1999 15:26:35 -0600
To: Keith Moore <[email-masked]>
From: Pete Resnick <[email-masked]>
Subject: Re: multipart + quoted-printable question
Cc: [email-masked], [email-masked]
Sender: [email-masked]

On 3/11/99 at 4:01 PM -0500, Keith Moore wrote:

> in other words, does the fact that the initial CRLF is attached to
> the boundary marker and not part of the preceding body part, mean
> that the preceding body part (in this case) doesn't end with a CRLF?

Yes, that is absolutely clear from the top of page 20 of RFC 2046, 
section 5.1.1.

> Now consider this body part (ending with a soft line break):
>
> --foo
> Content-transfer-encoding: quoted-printable
>
> =01=02=03=
> --foo
>
> is the canonical form of this body part also (hex) 01 02 03?
> or is this just illegal because there's no following CRLF?

I believe it's illegal. I don't see any ambiguity. There is no CRLF 
following "03=" in that body part.

pr
--
Pete Resnick <mailto:[email-masked]>
Eudora Engineering - QUALCOMM Incorporated
Ph: (217)337-6377 or (619)651-4478, Fax: (619)651-1102

To: [email-masked]
Subject: multipart + quoted-printable question
cc: [email-masked]
From: Keith Moore <[email-masked]>
Date: Thu, 11 Mar 1999 16:01:09 -0500
Sender: [email-masked]

I've come across what looks like an ambiguity in the MIME specs.
Consider the following bodypart within a multipart with boundary "foo":

--foo
Content-transfer-encoding: quoted-printable

=01=02=03
--foo

what is the canonical form of the body part ?

is it: (hex) 01 02 03 
or: (hex) 01 02 03 0D 0A ?

(I think it's the former)

in other words, does the fact that the initial CRLF is attached to 
the boundary marker and not part of the preceding body part, mean 
that the preceding body part (in this case) doesn't end with a CRLF?

Now consider this body part (ending with a soft line break):

--foo
Content-transfer-encoding: quoted-printable

=01=02=03=
--foo

is the canonical form of this body part also (hex) 01 02 03?
or is this just illegal because there's no following CRLF?

Keith

To: [email-masked]
Xref: clerew local.mime:119
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Thu, 11 Mar 1999 11:00:16 GMT
Lines: 115
Sender: [email-masked]

In <[email-masked]> Randall Gellens <[email-masked]> writes:

>At 1:59 PM +0000 3/5/99, Charles Lindsey wrote:

>The original version of the draft permitted either ">" or "> "; a 
>subsequent version used "> " only, and now we are at ">" only.

Actually, you could allow either by letting the dequote/destuff cycle
iterate until exhaustion.

>Since this is a new format, I think there are several advantages to 
>picking one canonical quote indicate and sticking to it.  Allowing 
>options, or making the quote indicator an additional C-T parameter, 
>adds complexity with very little benefit.

Indeed. The first quote character is under the control of the person who
generates the flowed format, so you _could_ insist on ">" as the only
supported possibility. However, the real problem is texts that come with a
variety of quoting standards, having lines beginning with ">|:" for 3
depths of quotation. Trouble is that many mailers allow you to choose your
quote character, and some people cannot resist the tmeptation to take full
advantage of all bells and whistles provided :-( . OTOH, I see no obvious
solution to that problem.

>Yes, it is a slim proposal, that solves mainly one problem.  That is 
>the goal.  I think it is very important to have something that is 
>very simple and can be quickly implemented.  Anything more complex 
>will be harder to get consensus on, and take longer to get it right 
>and then implemented and deployed.  So, I take your comment as 
>evidence that this is on the right track :-).

No, I still think it is too slim. Consider the range of possibilities that
might be provided:

1. Automatic re-wrapping of flowed lines. This is what is in your present
proposal.

2. Automatic rewrapping of indented lines.

    People like to indent portions of their_
    text so as to set it off from the_
    surrounding text.
This could be rewrapped as:
    People like to indent portions_
    of their text so as to set
    it off from the surrounding_
    text.
(observe my use of "_" to indicate a trailing space). This is a bit like
HTML <blockquote>, but more flexible. Note that you almost have a facility
like this insofar as it works with lines set off by ">>>>" in your
present proposal.

3. Automatic rewrapping of exdented lines.

  o People like to insert "bullet" symbols_
    to indicate important entry points in_
    their indented texts.
  o In addition, they do similar things using_
    numbers or letters (e.g. "(i)" or "a)",_
    but I won't bother to show an example of
    that.
Which would rewrap to:
  o People like to insert "bullet"_
    symbols to indicate important_
    entry points in their indented_
    texts.
  o In addition, they do similar_
    things using numbers or letters_
    (e.g. "(i)" or "a)", but I won't_
    bother to show an example of that.

4. Automatic bulleting or numbering of such indents (HTML <ul> or <ol>, or
even <dt>).

5. Centred text.

6. Headers of various levels
For display in larger fonts, or underlined with "---", "===", etc
according to level.

7. Bold/italic/underlining

And so on and so forth. If you do all those things (which I do not
suggest) then you have more-or-less arrived at Proletext.

The question is, how far to go? To my taste, the minumum that is worth
dowing is everything down to "2.", or perhaps "3.". Anything less is too
feeble to warrant introducing a new notation at all. Moreover, it should
be fairly easy to implement down to that level (though your space-stuffing
convention would need to be rethought). Not that implementation difficulty
should be a prime concern. If you are going to have to write programs to
do the job anyway, then writing a more complex program is not that much
extra overhead, and what I am suggesting is hardly rocket science.

>I did look at the proletext proposal, and it is interesting.  If 
>Format=Flowed is a success, I think it would be worthwhile to then 
>consider advancing proletext or something like it, as an additional 
>format type.  But I think we should get F=F deployed first.

Indeed so. It is somewhat bloated as it stands, but it could easily be
made into a reasonable text/proletext standard if a little restraint was
exercised.

And finally, I am concerned about the lack of mention of TAB characters.
If it is specifically your intention that they do not could as "SP",
either for flowing or for stuffing, then you should explicitly say so, to
remove all doubt. And BTW the ABNF accoring to the final RFC is "SPACE",
not "SP".

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

To: Randall Gellens <[email-masked]>
cc: [email-masked]
Subject: Re: draft-gellens-format-03.txt 
From: [email-masked]
Pgp-Action: PGP/MIME-signclear; rfc822=off; originator="<[email-masked]>"
Date: Thu, 11 Mar 1999 03:06:27 -0500
Sender: [email-masked]

On Wed, 10 Mar 1999 15:23:35 PST, you said:
> I did look at the proletext proposal, and it is interesting.  If 
> Format=Flowed is a success, I think it would be worthwhile to then 
> consider advancing proletext or something like it, as an additional 
> format type.  But I think we should get F=F deployed first.

Is the proletext proposal something that would be upward-compatible
with the current F=F draft?  Or can it be made so?  The last thing
we need to do is progress the current draft, and then go look
at a subtly different scheme from proletext - you *know* that at
least one major vendor will attempt to re-use the display engine
and Get It Very Wrong if it's incompatible. ;)

				Valdis Kletnieks
				Computer Systems Senior Engineer
				Virginia Tech

Date: Wed, 10 Mar 1999 15:23:35 -0800
To: [email-masked] (Charles Lindsey), [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Sender: [email-masked]

At 1:59 PM +0000 3/5/99, Charles Lindsey wrote:

> Although the usual way of quoting, when replying to news/email, is to stick
> a single '>' on the front, there are other possibilities, one of which is
> '> '. But the proposal as it stands will completely screw up on that. We
> need to think about this, because the present text of the usenet-format
> proposal actually recommends '> ' as an improvement over '>'. I daresay we
> could be talked out of that, but a safer way might be to have specific
> parameters which could be set in the Content-Type header to say what the
> quote-character and/or the stuffing-character should be.

The original version of the draft permitted either ">" or "> "; a 
subsequent version used "> " only, and now we are at ">" only.

Since this is a new format, I think there are several advantages to 
picking one canonical quote indicate and sticking to it.  Allowing 
options, or making the quote indicator an additional C-T parameter, 
adds complexity with very little benefit.

If we have one quote indicator, it makes the proposal a little easier 
to understand and implement, both in generation and reception.

If a receiving mailer understands Format=Flowed, it can display 
quoted text in any desirable way (current betas of Mac Eudora, for 
example, optionally display quoted flowed text using excerpt bars). 
If a receiving agent does not understand F=F, it appears as if the 
text was generating using the most popular quote indicator (">"). 
And I say this as someone who personally prefers "> ".

> Indeed, this whole proposal looks a little on the slim side - it solves
> one real problem, but only one. There is a scheme by Brad Templeton (which
> I have not looked at yet) on
> <http://www.templetons.com/tech/proletext.html> which you ought to look
> at. I gather that, by using mixed tabs and spaces at the end of a line, it
> can encode all sorts of behaviours (anything that can be expressed in
> binary), and even more if used on an empty line. And, like your scheme, it
> defaults to plain text if it breaks. It may indeed be more complex than is
> justified, but I think you should at least have a look.

Yes, it is a slim proposal, that solves mainly one problem.  That is 
the goal.  I think it is very important to have something that is 
very simple and can be quickly implemented.  Anything more complex 
will be harder to get consensus on, and take longer to get it right 
and then implemented and deployed.  So, I take your comment as 
evidence that this is on the right track :-).

I did look at the proletext proposal, and it is interesting.  If 
Format=Flowed is a success, I think it would be worthwhile to then 
consider advancing proletext or something like it, as an additional 
format type.  But I think we should get F=F deployed first.

Date: Wed, 10 Mar 1999 15:15:04 -0800
To: Dan Kohn <[email-masked]>, [email-masked]
From: Randall Gellens <[email-masked]>
Subject: Re: draft-gellens-format-03.txt
Cc: [email-masked]
Sender: [email-masked]

At 8:49 PM -0800 3/3/99, Dan Kohn wrote:

> A few small things:

Thanks for the comments.  I agree with them.
--
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Air Head Fired
--Newspaper headline

To: [email-masked]
Xref: clerew local.mime:116
Newsgroups: local.mime
Path: clerew!chl
From: [email-masked] (Charles Lindsey)
Subject: Re: draft-gellens-format-03.txt
Date: Fri, 5 Mar 1999 13:59:08 GMT
Lines: 33
Sender: [email-masked]

In <[email-masked]> Dan Kohn <[email-masked]> writes:

>The new draft looks good.  I think it's just about ready to be submitted
>to the IESG.

>A few small things:

A few more things.

Although the usual way of quoting, when replying to news/email, is to stick
a single '>' on the front, there are other possibilities, one of which is
'> '. But the proposal as it stands will completely screw up on that. We
need to think about this, because the present text of the usenet-format
proposal actually recommends '> ' as an improvement over '>'. I daresay we
could be talked out of that, but a safer way might be to have specific
parameters which could be set in the Content-Type header to say what the
quote-character and/or the stuffing-character should be.

Indeed, this whole proposal looks a little on the slim side - it solves
one real problem, but only one. There is a scheme by Brad Templeton (which
I have not looked at yet) on
<http://www.templetons.com/tech/proletext.html> which you ought to look
at. I gather that, by using mixed tabs and spaces at the end of a line, it
can encode all sorts of behaviours (anything that can be expressed in
binary), and even more if used on an empty line. And, like your scheme, it
defaults to plain text if it breaks. It may indeed be more complex than is
justified, but I think you should at least have a look.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Email:     [email-masked]  Web:   http://www.cs.man.ac.uk/~chl
Voice/Fax: +44 161 437 4506      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9     Fingerprint: 73 6D C2 51 93 A0 01 E7  65 E8 64 7E 14 A4 AB A5

From: Dan Kohn <[email-masked]>
To: [email-masked]
Cc: [email-masked]
Subject: draft-gellens-format-03.txt
Date: Wed, 3 Mar 1999 20:49:13 -0800 
Sender: [email-masked]

The new draft looks good.  I think it's just about ready to be submitted
to the IESG.

A few small things:

1)  In section 7.1, "if exactly one trailing space is added to one or
more lines of a message which uses the Format=Flowed parameter, the
effect may be a corrupted display or reply" is out of date.  Now that a
flowed line is defined as ending with 1*SP rather than SP, it is easier
for mailers that add trailing white space to corrupt the message.

2) non-sp        = %01-09 / %0B / %0C / %0E-19 / %21-7F

should be:

non-sp        = %01-09 / %0B-%0C / %0E-1F / %21-7F

(19 doesn't come before 20, 1F does.)

3) I would switch the order of 5.1 and 5.2, as a new reader is more
likely to get the concept if they first see how to generate flowed text
and then how to interpret it.

4) Before the insult example in section 5.5, use the fuller explanation
of soft and hard line breaks "(using '*' to indicate a soft line break,
that is, SP CRLF sequence, and '#' to indicate a hard line break, that
is, CRLF)".

From: "Larry Osterman (Exchange)" <[email-masked]>
To: "'Jacob Palme'" <[email-masked]>, IETF mailing list on e-mail headers <[email-masked]>
Subject: RE: Funny disruption of subject lines
Date: Wed, 3 Mar 1999 10:22:32 -0800 
Sender: [email-masked]

The multiple "Re:" may be caused by Exchange's localization rules for
subject prefixes (and yes, I know we don't do this according to 821bis, but
that prohibition was only added to 821bis a couple of months ago, and we've
been doing it for about 4 years)....

Exchange will cause long subject lines to be folded funny because of its
header wrapping rules - since it won't allow a header line to be longer than
72 characters (since there are mailers out there that badly choke on long
header lines), it scans back in the string looking for a "reasonable" place
to fold the header lines (it looks for a word break).  It will then fold the
subject using the RFC822 header line folding rules, which causes (this
example will probably not look right, because Exchange will wrap body text
at 72 characters, but the Subject: below is all on one line):

Subject: This is a message with an indescribably long subject.  I don't know
why anyone would ever want to have a subject that is this long, but there
are people who will want this.

Subject: This is a message with an indescribably long subject.  I 
        don't know why anyone would ever want to have a subject that 
        is this long, but there are people who will want this.

I believe that Exchange puts spaces in when it folds the lines, but it's
entirely possible that some client is changing the spaces into tabs (it's
all LWSP, right?).  What I cannot explain is the level of degradation that
your subject is showing, but it's possible you are seeing a combination of
these two effects (the "Re:" expansion and the long subject line expansion.

Larry Osterman
Sent from larryo-laptop.dns.microsoft.com running NT5 and Outlook 2000
and Exchange Server 5.5.  Please notify the sender of any difficulties

-----Original Message-----
From: Jacob Palme [mailto:[email-masked]]
Sent: Wednesday, March 03, 1999 3:39 AM
To: IETF mailing list on e-mail headers
Subject: Funny disruption of subject lines

I have noted that in some cases, the Subject line is split in funny ways.

For example, in a long message exchange with another person,
who has the following header in his message:
X-Mailer: Internet Mail Service (5.0.1458.49)
and where I have been using Eudora 4.0,

the Subject line started out like this (I have translated from
Swedish to English):

Subject: Speaker in the ending of Febr. or beginning of March 99??

But the Subject in the latest messages look like this:

Subject: Re: Re: Re: Re: Speaker in the end
	ing of  	febr. 	 or be  	ginning  	of March 99 
	??

The cause of the problem seems to be that some mailer has
split the subject into separate lines at arbitrary points,
probably because of the successive adding of "Re:" and then
merged them again, replacing the line breaks with tabs.
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

Date: Wed, 3 Mar 1999 12:38:40 +0100
To: IETF mailing list on e-mail headers <[email-masked]>
From: Jacob Palme <[email-masked]>
Subject: Funny disruption of subject lines
Sender: [email-masked]

I have noted that in some cases, the Subject line is split in funny ways.

For example, in a long message exchange with another person,
who has the following header in his message:
X-Mailer: Internet Mail Service (5.0.1458.49)
and where I have been using Eudora 4.0,

the Subject line started out like this (I have translated from
Swedish to English):

Subject: Speaker in the ending of Febr. or beginning of March 99??

But the Subject in the latest messages look like this:

Subject: Re: Re: Re: Re: Speaker in the end
	ing of  	febr. 	 or be  	ginning  	of March 99 
	??

The cause of the problem seems to be that some mailer has
split the subject into separate lines at arbitrary points,
probably because of the successive adding of "Re:" and then
merged them again, replacing the line breaks with tabs.
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme