Subject: Re: comments on draft-crocker-email-arch-01
From: Tony Finch <[email-masked]>
To: [email-masked]
Date: Thu, 30 Dec 2004 23:50:13 +0000
List-Id: <ietf-smtp.imc.org>

On Thu, 23 Dec 2004, Bruce Lilly wrote:
>
> What about the Return-Path header field?  It is explicitly defined
> as containing the content of the SMTP MAIL FROM command when an
> SMTP receiver makes "final" delivery.

Yes, and the Received: line sometimes contains a copy of the RCPT TO
command (when there's only one). These are copies of parts of the
envelope for future reference, not the envelope itself.

> I understand that there is a desire to distinguish content
> following SMTP DATA from other SMTP command content, but SMTP
> itself broke the analogy to postal mail by requiring (e-)mail
> handlers to scribble on the message content.

Analogies between the software world and the real world are usually
not perfect.

> The draft under discussion is about the overall architecture
> rather than a nitty-gritty detail peculiar to SMTP
> protocol.  From an architectural point of view it is
> important to distinguish the end-to-end, author/sender-to-
> recipient information from ancillary notations (to
> specify who/what may set/read which portions of a message),

I agree that this is necessary. However you cannot use the term "envelope"
for the transport-related header fields because "envelope" already means
something else in the context of Internet email. If you want to subdivide
the message header, please choose new terms to describe its parts.

This is an important architectural misfeature of Internet email. It is
best to talk about Internet email as it is, and not to talk about it by
shoehorning it into some idealization that doesn't apply in the real
world.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
PORTPATRICK: ON SATURDAY ALL AREAS WILL HAVE STRONG OR GALE FORCE SOUTHWEST
WINDS POSSIBLY TOUCHING SEVERE GALE FORCE AT FIRST OVER FAIR ISLE FAEROES AND
SOUTHEAST ICELAND.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: comments on draft-crocker-email-arch-01
Date: Sat, 25 Dec 2004 04:03:56 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

For what its worth, Isn't the Return-Path: (like the old UUCP "From :")
considered "intermediate" network control header lines?

In other words, SMTP adds it for a Final Delivery or Local hosting domain,
but once the gate or server-side host mail importer deals with it, it is
removed.

This is how our system works, and I believe most work this way as well.
However I do see some systems retain the now useless Return-Path for user
mail pickup.

Example, with SLIP/UUCP,  a "From " was added to address bounce situations:

        SLIP/UUCP (From Added)
            -->  GATEWAY
                ---> USER STORAGE  (From Removed)
                ---> DOWNLINK HOSTING STORAGE  (From kept)
                ---> BOUNCE (From utilized for new notification)

This network control header line was removed once the gate determined a
valid final destination.

Similarly with SMTP:

       SMTP (Return-Path Added)
            --> GATEWAY
                  ---> USER STORAGE  (Return-Path Removed)
                  ---> BOUNCE (Return-Path utilized  for new notification)

Also I think it is a matter of implementation as well, on how the server
stores its mail and outbound queues.

For example:  A system can retain the transport information in a local
storage format such as:

    X-MAIL FROM:
    X-RCPT TO:
    X-DATA: [delimiter]
    Return-Path:  [only for local delivery]
    Received:
    Actual Email Data

The gate now handles this accordingly:

1) SMTP receives MDA or authorized relay mail   The above format go into a
common inbound queue,

2) The SMTP router eyeballs the inbound queue to determine local vs. relay,

3) For Local, the gate takes the X-DATA block, strips Return-path:, imports
for user.  A local validation is repeated. This allows the gate to create a
bounce just in case the SMTP process did not do it (bad practice today).

4) For Route, there is no Return-Path, and the relay will use the X- data
for the next SMTP send session.

The above is our current design for the Return-Path since 1996.  It was
never designed to be passed on to the user.

Happy Holidays,

---
Hector Santos
WINSERVER "Wildcat! Interactive Net Server"
http://www.santronics.com

----- Original Message -----
From: "Bruce Lilly" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, December 23, 2004 9:34 PM
Subject: Re: comments on draft-crocker-email-arch-01

>
> On Thu December 23 2004 15:04, Tony Finch wrote:
>
> > I agree that the 822 message header is uncomfortably stuck doing two
jobs,
> > but I disagree that this implies that the term "envelope" should include
> > any header fields.
>
> What about the Return-Path header field?  It is explicitly defined
> as containing the content of the SMTP MAIL FROM command when an
> SMTP receiver makes "final" delivery.  What about RFC 3939
> Caller-ID, which is explicitly derived from handling information?
> Neither was placed in the message content by the originator;
> they are annotations made by transport processes.
>
> I understand that there is a desire to distinguish content
> following SMTP DATA from other SMTP command content, but SMTP
> itself broke the analogy to postal mail by requiring (e-)mail
> handlers to scribble on the message content (Mail-From and
> Return-Path fields)[*].  It is desirable to be able to distinguish
> end-to-end content (set by author and/or sender and intended
> for the recipient(s)) from postmarks and other bits of
> scribbling made during transport -- by continuation of the
> analogy to postal mail, those postmarks and other notations
> are placed on the envelope.  The poster child case for making
> the difference clear is the Reply-To field, which is clearly
> intended as an end-to-end originator-specified field, but
> which some errant transport entities have been known to
> scribble upon.
>
> The draft under discussion is about the overall architecture
> rather than a nitty-gritty detail peculiar to SMTP
> protocol.  From an architectural point of view it is
> important to distinguish the end-to-end, author/sender-to-
> recipient information from ancillary notations (to
> specify who/what may set/read which portions of a message),
> and envelope/header/body work well as analogous terms;
> the envelope is where transport-related scribbling (including
> delivery status notification requests) is placed, the
> header contains information set by the author, intended
> for the recipient (which usually matches the envelope
> instructions for delivery and indication of source, but
> need not do so and after submission do not in fact affect
> delivery) and is structured, while the body is free-form
> content intended for end-to-end use.
>
> The practice of transport processes scribbling envelope-
> type markings on the message content (the header) began
> with SMTP and is peculiar to SMTP; it was not so with FTP
> or MTP, and is not so with UUCP mail transport.  Although
> SMTP is certainly an important protocol, I believe we
> should not confuse its peculiarities with architectural
> principles.
>
>
> * and that bad practice has been adopted subsequently
> by others, unfortunately
>
>

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: comments on draft-crocker-email-arch-01
Date: Thu, 23 Dec 2004 21:34:46 -0500
Sender: [email-masked]

On Thu December 23 2004 15:04, Tony Finch wrote:

> I agree that the 822 message header is uncomfortably stuck doing two jobs,
> but I disagree that this implies that the term "envelope" should include
> any header fields.

What about the Return-Path header field?  It is explicitly defined
as containing the content of the SMTP MAIL FROM command when an
SMTP receiver makes "final" delivery.  What about RFC 3939
Caller-ID, which is explicitly derived from handling information?
Neither was placed in the message content by the originator;
they are annotations made by transport processes.

I understand that there is a desire to distinguish content
following SMTP DATA from other SMTP command content, but SMTP
itself broke the analogy to postal mail by requiring (e-)mail
handlers to scribble on the message content (Mail-From and
Return-Path fields)[*].  It is desirable to be able to distinguish
end-to-end content (set by author and/or sender and intended
for the recipient(s)) from postmarks and other bits of
scribbling made during transport -- by continuation of the
analogy to postal mail, those postmarks and other notations
are placed on the envelope.  The poster child case for making
the difference clear is the Reply-To field, which is clearly
intended as an end-to-end originator-specified field, but
which some errant transport entities have been known to
scribble upon.

The draft under discussion is about the overall architecture
rather than a nitty-gritty detail peculiar to SMTP
protocol.  From an architectural point of view it is
important to distinguish the end-to-end, author/sender-to-
recipient information from ancillary notations (to
specify who/what may set/read which portions of a message),
and envelope/header/body work well as analogous terms;
the envelope is where transport-related scribbling (including
delivery status notification requests) is placed, the
header contains information set by the author, intended
for the recipient (which usually matches the envelope
instructions for delivery and indication of source, but
need not do so and after submission do not in fact affect
delivery) and is structured, while the body is free-form
content intended for end-to-end use.

The practice of transport processes scribbling envelope-
type markings on the message content (the header) began
with SMTP and is peculiar to SMTP; it was not so with FTP
or MTP, and is not so with UUCP mail transport.  Although
SMTP is certainly an important protocol, I believe we
should not confuse its peculiarities with architectural
principles.

* and that bad practice has been adopted subsequently
by others, unfortunately

From: "Hector Santos" <[email-masked]>
To: "John Leslie" <[email-masked]>, "Tony Finch" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: comments on draft-crocker-email-arch-01
Date: Thu, 23 Dec 2004 21:20:35 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message -----
From: "John Leslie" <[email-masked]>
To: "Tony Finch" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, December 23, 2004 5:13 PM
Subject: Re: comments on draft-crocker-email-arch-01

>    I find Tony's usage far more comfortable -- in that I can actually
> explain it to someone and hope to get the idea across. IMHO, Dave is too
> hung up on parallels to postal mail, and is thinking "envelope
information"
> should mean anything which would normally be recorded on the envelope.
> Personally, I'd prefer to reserve "envelope" for Tony's meaning, and coin
> another term -- perhaps something akin to "routing information" for Dave's
> meaning.
>
>    Obviously, everone's mileage-may-vary...
>
>    But I think there's a very good case for defining separate terms for
> these two meanings.

Taken into account other networks we have experience in, "Network Control
Lines or Transport Information," were the technical terms used.  Envelope
was used more for the layman.

Fidonet, QWK, UTI, and the Internet all share the common layout.  Some are
binary versus text based.  But they all had:

    Network Control lines
    Header Lines
    Message Body

The internet RFC 2821/2882 I believe calls the Network control line "trace
information."

The Network Control lines usually did match the transport information and
this is also true with SMTP in the case that we add two transport lines to
the network control block:

    Return-Path
    Received:

Fidonet, for example, added:

    ^aPID:
    ^aTID:
    ^aREPLY:

PID/TID (Process and transaction ID) is a skin to a Received line. and the
Reply was the "return path."   We called these Network control information.

Since Fidonet had a binary format, control lines were not limited to the top
of the blocks. It could be embedded it th body, but in general, it was the
TOP and BOTTOM of a message block:

For example, in a Fidonet group mail disitrbution (i.e, News for the
Internet), these were added to the bottom of each message as it was streamed
thru the network:

    ^aPath:  Host tracing
    ^aSeen-By: Nodes with Host tracing

This allowed for tracing and also dupe processing. It was serious hanging
offense to create dupes in the network.

In any case, I like Transport information for SMTP and Network Control lines
for headers added to the RFC x822 header that is part of the distribution,
not necessary concerned for the user consumption.

Sincerely,

Hector Santos, CTO
Santronics Software, Inc.
http://www.santronics.com
305-431-2846 Cell
305-248-3204 Office

Date: Thu, 23 Dec 2004 17:13:45 -0500
From: John Leslie <[email-masked]>
To: Tony Finch <[email-masked]>
Cc: [email-masked]
Subject: Re: comments on draft-crocker-email-arch-01
Sender: [email-masked]

Tony Finch <[email-masked]> wrote:
> On Wed, 22 Dec 2004, Bruce Lilly wrote:
>>
>> Yes.  I remarked on that a half-year ago on the mail-ng
>> list (http://www.imc.org/mail-ng/mail-archive/msg00715.html),
>> and the issue came up recently on the ietf-822 list
>> (http://www.imc.org/ietf-822/mail-archive/msg05290.html).
> 
> I agree that the 822 message header is uncomfortably stuck doing two jobs,
> but I disagree that this implies that the term "envelope" should include
> any header fields.

   Tony, like many of us, has acquired a habit of referring to the STMP
stuff that comes before the DATA command as "envelope" information.

   Dave's draft gives a quite different meaning to "envelope":
] 
] 5.1 Envelope
] 
] Information that is directly used or produced by the email transfer
] service is called the "envelope". It controls and  records handling
] activities by the transfer service. Internet mail has a fragmented
] framework for handling this "handling" information. The envelope exists
] partly in the transfer protocol SMTP [RFC2821] and partly in the message
] object [RFC2822].
] 
] Direct envelope addressing information, as well as optional transfer
] directives, are carried in-band by MTAs. All other envelope information,
] such as trace records, is carried within the content headers. Upon
] delivery, SMTP-level envelope information is typically encoded within
] additional content  headers, such as Return-Path and Received (From
] and For).

   I find Tony's usage far more comfortable -- in that I can actually
explain it to someone and hope to get the idea across. IMHO, Dave is too
hung up on parallels to postal mail, and is thinking "envelope information"
should mean anything which would normally be recorded on the envelope.
Personally, I'd prefer to reserve "envelope" for Tony's meaning, and coin
another term -- perhaps something akin to "routing information" for Dave's
meaning.

   Obviously, everone's mileage-may-vary...

   But I think there's a very good case for defining separate terms for
these two meanings.

--
John Leslie <[email-masked]>

Date: Thu, 23 Dec 2004 20:04:46 +0000
From: Tony Finch <[email-masked]>
To: [email-masked]
Subject: Re: comments on draft-crocker-email-arch-01
Sender: [email-masked]

On Wed, 22 Dec 2004, Bruce Lilly wrote:
>
> Yes.  I remarked on that a half-year ago on the mail-ng
> list (http://www.imc.org/mail-ng/mail-archive/msg00715.html),
> and the issue came up recently on the ietf-822 list
> (http://www.imc.org/ietf-822/mail-archive/msg05290.html).

I agree that the 822 message header is uncomfortably stuck doing two jobs,
but I disagree that this implies that the term "envelope" should include
any header fields.

I did a more thorough survey of RFCs related to internet email (I can send
people my notes if they want). I excluded X.400 and other non-internet
protocols, and MIME types not related to the transport - so I included
DSNs but excluded S/MIME (which only uses "envelope" to talk about
cryptographic encapsulation).

The first time the envelope is defined as the MAIL FROM and RCPT TO
commands is in RFC 976's definition of batch SMTP; however it's also
implicit in RFC 822.

The first time the SMTP envelope is properly defined is in RFC 1123. There
are 50ish email-related RFCs in the 15 years since then which mention the
word "envelope" and the VAST MAJORITY of them either explicitly or
implicitly use the same definition.

The exceptions include the IMAP RFCs (because of the name of an IMAP
protocol element which substantially predates 1123) and a few instances of
sloppy wording.

In RFC 2130 "envelope" is being used in the general sense of protocol
encapsulation - a place where you might include the character set
information related to the encapsulated data - and MIME encapsulation is
used as an example. This isn't strictly email-related so isn't really a
counter-example.

In RFCs 2442 and 2554 "envelope" is used where it would be more correct to
say "envelope and header". This is probably just an omission. In both
cases they are talking about security of message transport protocol
elements which are harder to secure than the message header.

Having said that I excluded X.400 RFCs (which exclusively use the term
"envelope" to refer to X.400 protocol elements, and are generally
abstracted away from SMTP) there's a useful comment in RFC 1506:
 - RFC 1327 doesn't define how to map the MTS.OriginatorName and
   the MTS.RecipientName (often referred to as the P1.originator
   and P1.recipient), since this depends on which underlying 822-
   MTS is used. In the very common case that RFC 821 (SMTP) is
   used for this purpose, the mapping is normally as follows:
      MTS.Originator-name <->   MAIL FROM:
      MTS.Recipient-name  <->   RCPT TO:
A lot of X.400 envelope information has to be put into the 822 header when
gatewaying because of the difference in architecture between X.400 and
internet email, but I don't see why this should have any bearing on the
meaning of "envelope" in the internet world - and indeed it doesn't seem
to have had any, judging by my surveys.

So I still think there is no lack of consensus in the meaning of
"envelope" in the context of internet email, and this has been the
case for at least 15 years.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
BERWICK ON TWEED TO WHITBY: WEST 6 TO GALE 8 BACKING SOUTHWEST 4 OR 5 LATER.
RAIN THEN SHOWERS. MODERATE OR GOOD. MODERATE OR ROUGH.

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Dave Crocker <[email-masked]>, [email-masked]
Subject: Re: comments on draft-crocker-email-arch-01
Date: Wed, 22 Dec 2004 19:20:11 -0500
Sender: [email-masked]

On Wed December 22 2004 17:17, Dave Crocker wrote:

> >  To
> >  summarize the following, it seems to me that "envelope" in the context of
> >  internet email always means the MAIL and RCPT commands preceding DATA, and
> >  does not include any of the header. It's only when you stray into
> >  gatewaying and other non-internet email protocols that the line gets
> >  blurred.
> 
> Not quite "only", as I'll note below.  Major point was that this is an issue that has been at best extremely muddled over the life of Internet history. Periodically, we get lost in the muddle of it.

Yes.  I remarked on that a half-year ago on the mail-ng
list (http://www.imc.org/mail-ng/mail-archive/msg00715.html),
and the issue came up recently on the ietf-822 list
(http://www.imc.org/ietf-822/mail-archive/msg05290.html).

> So, I'm not kidding. We really screwed up on this issue.  

Only about a week left to catch the "understatement of
the year" prize :-)
 
> Over the years the problem has come up periodically, but never got resolved in any definitive way (or at all.)  I'd like the architecture document to lock down the concept of envelope more precisely.  That is why I am suggesting the definition:
> 
> 				The envelope comprises handling information that is created 
> 				for consumption by the handling service or created by the 
> 				handling service, primarily for consumption y email operators 
> 				or for operational analysis.

That would certainly include Return-Path (which is
supposed to be generated from MAIL FROM on "final"
delivery in any case) and Received (nee Mail-From)
fields. Also Disposition-Notification-{To, Options},
the newfangled Caller-ID, etc.
  
> Just to quibble, RFC 733 was 822's predecessor.  
> 
> Everything before 733 was folks' personal comments (except, of course, the original FTP specification.)  822 was the first email 'standard'.

Well, as long as we're in quibbling mode... while
822 is the current full Standard, 733's title says
it's a standard (the RFC Index says status is UNKNOWN),
and RFC 561 (the first in the 561-680-724-733-822-2822
chain of message format RFCs) speaks of standardization.
On the transfer protocol side of the aisle, 821 is a
full standard, whereas its SMTP predecessor, RFC 788,
has status unknown and doesn't claim to be a standard
(merely a specification of protocol). Its non-simple
predecessors, RFCs 780 and 772 have similar status and
character. Likewise for the versions of FTP (RFC 765
and earlier) which included the MAIL and/or MLFL
commands.

Date: Wed, 22 Dec 2004 23:10:01 +0000
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked]
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1870869256-550541466-1103757001=:17125
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE

> >=A0 RFC 822 explicitly states that it does not have anything to do with =
the
> >=A0 envelope.
>
> Well, not quite.  Yes, it has an initial statement to that effect, but th=
en:
>
> > 4.7.3. =A0ENCRYPTED
> ....
> > Note: Unfortunately, headers must contain envelope, as well
> > as contents, information.

I think that's referring back to the following text, which is about
message submission:

          However, some message systems may use information  from  the
     contents  to create the envelope.  It is intended that this stan-
     dard facilitate the acquisition of such information by programs.

> =09=09The envelope comprises handling information that is created
> =09=09for consumption by the handling service or created by the
> =09=09handling service, primarily for consumption y email operators
> =09=09or for operational analysis.

I don't think that's a useful definition because it leaves us without a
term to describe what all the SMTP-related RFCs call the envelope, and
it overlaps with the meaning of "trace fields" which already has a term of
its own. Your expanded definition might be useful in a broader context but
it doesn't accurately fit the way Internet email works. I thought that
non-Internet email was out of scope for this document.

> > Legal disclaimers and ads are often added by border-MTA manglers (e.g.
> > http://www.goldmark.org/jeff/stupid-disclaimers/). I was not thinking o=
f
> > mailing lists exclusively - I was trying to be as broad as possible.
>
> I'm inclined to class dislcaimer-type information as representing local
> policy, so that their attachment comes under the umbrella of MSA.
> That's a bit of a reach, but I do not think it unreasonable.

Border MTAs deal with incoming as well as outgoing email. Another kind of
mangling which might be more likely on an incoming border MTA is the
addition of a blurb saying that the message has been scanned for viruses.
In no way could this be considered an MSA function.

Tony.
--=20
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOUTH UTSIRE FORTIES: WEST OR NORTHWEST BACKING SOUTHWEST 7 TO SEVERE GALE =
9,
VEERING WEST 6 TO GALE 8 LATER. RAIN AT TIMES. MODERATE OR GOOD OCCASIONALL=
Y
POOR.
--1870869256-550541466-1103757001=:17125--

From: Dave Crocker <[email-masked]>
To: Tony Finch <[email-masked]>
CC: <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Wed, 22 Dec 2004 14:17:54 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

Tony,

>  To
>  summarize the following, it seems to me that "envelope" in the context of
>  internet email always means the MAIL and RCPT commands preceding DATA, and
>  does not include any of the header. It's only when you stray into
>  gatewaying and other non-internet email protocols that the line gets
>  blurred.

Not quite "only", as I'll note below.  Major point was that this is an issue that has been at best extremely muddled over the life of Internet history. Periodically, we get lost in the muddle of it.

>  RFC 2821 (SMTP) and its predecessors RFC 1123 (application requirements)

RFC2821 introduces the term as "SMTP envelope".  My guess is that this was not meant to suggest that the usage was peculiar to SMTP, but still, it's careful to delimit the scope of the term.

More generally, yes, all of the transmission-related specs view the issue as transmission-information versus non-trasmission information.

>  RFC 822 explicitly states that it does not have anything to do with the
>  envelope. 

Well, not quite.  Yes, it has an initial statement to that effect, but then:

>  4.7.3.  ENCRYPTED
....
>  Note:  Unfortunately, headers must contain envelope,  as  well
>  as  contents,  information.  

So, I'm not kidding. We really screwed up on this issue.  

Over the years the problem has come up periodically, but never got resolved in any definitive way (or at all.)  I'd like the architecture document to lock down the concept of envelope more precisely.  That is why I am suggesting the definition:

				The envelope comprises handling information that is created 
				for consumption by the handling service or created by the 
				handling service, primarily for consumption y email operators 
				or for operational analysis.

>Its predecessor RFC 724 (before SMTP) describes the concept of

Just to quibble, RFC 733 was 822's predecessor.  

Everything before 733 was folks' personal comments (except, of course, the original FTP specification.)  822 was the first email 'standard'.

>  The X.400 gatewaying RFCs (2156 etc.) describe a rather more elaborate
>  envelope than SMTP. 

I think that the main lesson from gatewaying specs is to demonstrate that envelope information is broader than the SMTP commands.

> > >  "gateway"/"firewall":
>  >
> > My own preference is to have a single manglers, and then try to define
> > sub-categories.  So, a security gateway is a firewall.  a mail protocol
> > gateway is something like x.400/internet.
>
>  I think that gateways to other messaging environments are rather different
>  from manglers within an internet system, because the former try to
>  preserve semantics whereas the latter deliberately do not.

interesting point.

> >  adding footers, such as list unsubscription instructions or stupid legal
> > >  disclaimers or ads
>  >
> > well, that's not a firewall.  that's a mailing list and i model that as
> > a separate user, creating a new message...
>
>  Legal disclaimers and ads are often added by border-MTA manglers (e.g.
>  http://www.goldmark.org/jeff/stupid-disclaimers/). I was not thinking of
>  mailing lists exclusively - I was trying to be as broad as possible.

I'm inclined to class dislcaimer-type information as representing local policy, so that their attachment comes under the umbrella of MSA.  That's a bit of a reach, but I do not think it unreasonable.

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: "Willemien" <[email-masked]>
To: "Dave Crocker" <[email-masked]>, <[email-masked]>
Subject: [Another long] commnets on draft-crocker-email-arch-01
Date: Wed, 22 Dec 2004 17:21:28 -0000
Sender: [email-masked]

First of all I want to praise your work. It is useful to have such a draft.

I am not an email specialist so please check everything thoroughly before
accepting.

Also at the end became a little messy, but I still hope you can use it.

Missing points

The draft is very SMTP centred it would be better to integrate other email
systems in it.

I propose to put this discussion also in the mail-NG mailing list
It would be nice to have one overarching mail-architecture for currently
used mail systems.
(including mail systems that are being developed)

The draft does not mention the
(RFC 3888) Tracking User Agent (TUA) and the Tracking Server

Some More detailed comments

MTA -- MTA interchange with errors

the SMTP errors are missing
should be something like

MSA <---+
        |
 :      | dsn (delivery status notification)
MTA--->-+
 | -<+
 |   |
 V   | SMTP error (500+)
MTA -+
(HM drawing in ASCII is not my first choice)

Different MTA's

There are architectional differences between MTA's

I would suggest the following sub categories

rMTA an MTA that is named in the receivers MX list

fMTA
the final MTA deserves a special place (This is the MTA
where the lowest MX records refer to, explain that there can be more than
one)

Message Tracking Model  (RFC 3888) also uses
Intermediate Message Transfer Agent (MTA)
Foreign Message Transfer Agent
Gateway Message Transfer Agent (GW-MTA)

I myself was more thinking about  a catogarisation of MTA's on the bases if
whether they have or have not a RDNS, whois, MX or other DNS record.

Message stores

I think there is reason to put an oMS (originator message store) between the
oMUA and MSA. MUA's store their message and it looks functualy better to
actually place a message store there.
(messages in this store possibly have a Bcc header)

The envelope ends at the MDA or MS.
But already long before that the envelope has  been shrinking. Only
recipients that are in the downstream part of the MTA-MTA link stay in the
envelope.

Expandable architecture
Structures in the real world are always more complex than the structures on
paper.
But this draft should have the structural elements to describe them.
Lets start with:
A personal problem: our system does not fit in the architecture

We have the system that our mail is collected by a catch-all POP3 mailbox at
our ISP.
Our mailserver in its turn collects the POP3 mail and redistributes it to
other POP3 mailboxes.
How should I call our mailserver according to this architecture?

An Catch all mailbox in this situation needs special care because we need
the SMTP envelop to redistribute the email properly. Only very few ISPs
acknowledge that. An catch-all mailbox in this situation is more than just a
anything alias.

(This is only one example for expanding figure 2 but figure 2 should be
expandable to include structures like mine that actually exist in the real
world)

Comment about figure 1

Why are providers mentioned?
They do not seem to have any particular function.
If they have different architectual functions they should have different
names.

Levels in mail architecture

There are some levels distinguishable in mail architecture

on the top is the user (sender) --user (receiver) level, mail is made to
give a message from one human user to another human user.

below that is the

MUA -- MUA level this level is about MIME encryption  etc.

below that is the

MSA -- MDA level

Some levels lower are the
bottom levels

MUA --> MS --> MSA
MSA --> MTA
MTA --> MTA
MTA --> MDA
MDA --> MS --> MUA

I haven't filled in all levels but it seems a handy instrument to look and
name them in  the architecture.

Dictionary

A dictionary from "old" names to the new names would be useful
examples (This is definitely not a complete reference)

UA is used in many RFC's but in this one it is called a MUA

A Local Delivery Agent (LDA) RFC 3888 is now called a Mail Delivery Agent
(MDA)

I hope you can use my comments.

From: "Willemien" <[email-masked]>
To: "Dave Crocker" <[email-masked]>, <[email-masked]>
Subject: [Another long] commnets on draft-crocker-email-arch-01
Date: Wed, 22 Dec 2004 15:51:51 -0000
Sender: [email-masked]

 First of all I want to praise your work. It is useful to have such a draft.

I am not an email specialist so please check everything thourougly before
accepting.

Also at the end became a little meshy, but i still hope you can use it.

Missing points

The draft is very SMTP centred it would be better to integrate other email
systems in it.

I propose to put this discussion also in the mail-NG mailing list
It would be nice to have one overarching mail-architecture for currently
used mail systems.
(including mail systems that are being developed)

The draft does not mention the
(RFC 3888) Tracking User Agent (TUA) and the Tracking Server

Some More detailed comments

MTA -- MTA interchange with errors

the SMTP errors are missing
should ne something like

MSA <---+
        |
 :      | dsn (delivery status notification)
mta--->-+
 | -<+
 |   |
 V   | SMTP error (500+)
MTA -+

(Hm drawing in ascii is not my first choice)

Different MTA's

There are architectional differences between MTA's

I would suggest the following sub catagories

rMTA an MTA that is named in the receivers MX list

fMTA
the final MTA deserves a special place call it the fMTA (This is the MTA
where the lowest MX records refer to, explain that there can be more than
one)

Message Tracking Model  (RFC 3888) also uses uses
Intermediate Message Transfer Agent (MTA)
Foreign Message Transfer Agent
Gateway Message Transfer Agent (GW-MTA)

I myself was more thinking about  a catogarisation of MTA's on the bases if
they have an RDNS, whois, MX or other DNS record.

Message stores

I think there is reason to put an oMS (originator message store) between the
oMUA and MSA. MUA's store their message and it looks functualy better to
actually place a message store there.
(messages in this store possibly have a Bcc header)

The envelope ends at the MDA/MS.
But allready long before that the enveloppe has  been shrinking. Only
recipients that are in the downstream part of the MTA-MTA link stay in the
envelope.

Expandable architecture
Structures in the real wor;d are always more complex than the structures on
paper.
But this draft should have the structural elements to discribe them.
Lets start with:
A personal problem: our system does not fit in the architecture

We have the system that our mail is collected by a catch-all POP3 mailbox at
our ISP.
Our mailserver in its turn collects the POP3 mail and redistributes it to
other POP3 mailboxes.
How should I call our mailserver according to this architecture?

An Catch all mailbox in this situation needs special care because we need
the SMTP envelop to redistribute the email properly. Only very few ISPs
acknowledge that. An catch-all mailbox in this situation is more than just a
anything alias.

(This is only one example for expanding figure 2 but figure 2 should be
expandable to include structures like mine that actually exist in the real
world)

Comment about figure 1

Why are providers mentioned?
They do not seem to have any particular function.
If they have different architectual functions they should have different
names.

Levels in mail architecture

There are some levels distinguisable in mail architecture

on the top is the user (sender) --user (receiver) level, mail is made to
give a message from one human user to another human user.

below that is the

MUA -- MUA level this level is about MIME encryption  ect.

below that is the

MSA -- MDA level

Some levels lower are the
bottum levels

MUA --> MS --> MSA
MSA --> MTA
MTA --> MTA
MTA --> MDA
MDA --> MS --> MUA

I haven't filled in all levels but it seems a handy instrument to look and
name them in  the architecture.

Dictionary

A dictionary from "old" names to the new names would be useful

UA is used in many RFC's but in this one it is called a MUA

a Local Delivery Agent (LDA) RFC 3888 is now called a Mail Delivery Agent
(MDA)

I hope you can use my comments.

Date: Wed, 22 Dec 2004 12:30:37 +0000
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked]
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1870869256-565155424-1103718637=:24304
Content-Type: TEXT/PLAIN; charset=ISO-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE

> >=A0 I'm inclined to talk about message submission in the abstract, with =
some
> >=A0 functions typically implemented by the MUA (e.g. preparing the envel=
ope)
> >=A0 and some by the MSA (header fix-ups and policy enforcement).
>
> well, the intent was to permit the exact venue for some of these
> functions (mua vs. msa) to be fluid.)

I agree, though I'm not sure this comes across very clearly in the draft.

> >=A0 I conclude that there's no detailed, precise definition of which par=
ts of
> >=A0 the message header might constitute "handling information", and migh=
t
> >=A0 therefore constitute part of the Crocker-envelope. The Finch-envelop=
e, on
> >=A0 the other hand, is well-defined and commonly understood.
>
> not by crocker (...)

Just a convenient label :-)

> I had not thought of my assessment on this as being particular to me,
> although i know there is variation within the community.  I do not have
> any sense of a solid consensus within the community.  If I did, I'd use
> it.

I don't think I have ever seen "envelope" used to include parts of the
header before I read your document. Hmm, time for a survey I think. To
summarize the following, it seems to me that "envelope" in the context of
internet email always means the MAIL and RCPT commands preceding DATA, and
does not include any of the header. It's only when you stray into
gatewaying and other non-internet email protocols that the line gets
blurred.

RFC 2821 (SMTP) and its predecessors RFC 1123 (application requirements)
and RFC 1869 (SMTP service extensions), RFCs 3461/3464 etc. (delivery
status notifications), RFC 2076 (common header fields), RFCs 3028 etc.
(sieve) all clearly use the "not DATA" envelope. RFC 2476 (submission),
RFC 3834 (auto-responses) implicitly use the same definition.

RFC 822 explicitly states that it does not have anything to do with the
envelope. Its predecessor RFC 724 (before SMTP) describes the concept of
the envelope, using examples analogous to the SMTP MAIL and RCPT commands.
It says that "envelope-like" information may be attached to the message
header but this is evidently not strictly part of the envelope.

The X.400 gatewaying RFCs (2156 etc.) describe a rather more elaborate
envelope than SMTP. They describe the same layering of envelope and data,
the latter comprising header and body. The main difference (as I
understand it from a brief glance) is that the more elaborate X.400
envelope has space for trace fields, which internet email puts in the
message header instead.

RFC 1711 (classifications in email routing) leaves the word "envelope"
deliberately vague so that it can be applied to seven different early
1990s email systems.

IMAP (RFC 3501 etc.) is an anomaly: it has its own "envelope" which is
explicitly not the same as the SMTP envelope. The IMAP envelope is a
subset of the message header, comprising "user-to-user" fields which have
very little to do with message transmission; therefore they are not an
envelope in the usual sense. (The fields are: date, subject, from, sender,
reply-to, to, cc, bcc, in-reply-to, and message-id.)

> >=A0 "gateway"/"firewall":
>
> My own preference is to have a single manglers, and then try to define
> sub-categories.  So, a security gateway is a firewall.  a mail protocol
> gateway is something like x.400/internet.

I think that gateways to other messaging environments are rather different
from manglers within an internet system, because the former try to
preserve semantics whereas the latter deliberately do not.

> >=A0 adding footers, such as list unsubscription instructions or stupid l=
egal
> >=A0 disclaimers or ads
>
> well, that's not a firewall.  that's a mailing list and i model that as
> a separate user, creating a new message...

Legal disclaimers and ads are often added by border-MTA manglers (e.g.
http://www.goldmark.org/jeff/stupid-disclaimers/). I was not thinking of
mailing lists exclusively - I was trying to be as broad as possible.

Tony.
--=20
f.a.n.finch  <[email-masked]>  http://dotat.at/
LYME REGIS TO LANDS END INCLUDING THE ISLES OF SCILLY: WEST 5 TO 7. RAIN
CLEARING LATER. GOOD OR MODERATE. ROUGH TO VERY ROUGH BECOMING VERY ROUGH T=
O
HIGH.
--1870869256-565155424-1103718637=:24304--

From: Dave Crocker <[email-masked]>
To: Bruce Lilly <[email-masked]>, <[email-masked]>
CC: Tony Finch <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Tue, 21 Dec 2004 21:44:35 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

On Tue, 21 Dec 2004 19:27:18 -0500, Bruce Lilly wrote:
>  On Tue December 21 2004 19:02, Dave Crocker wrote:
>
> > what's the problem with mere enforcement?  why must the msa create fields?
>
>  E.g. to add the mandatory Date header field if missing
>  (the semantics may be wrong (submission time rather than
>  creation time)) (the MUA might have no reliable clock
>  from which it could generate date-time).

this does not require that the msa create the field.  if an acceptable field is not present, the msa could merely reject the message.

my point is that the degree of creativity for an msa is legitimately highly variable and depends entirely upon the site policies.  there is nothing in the abstraction of Internet mail that dictates narrower and more rigid assignments of data creation.

if folks have a different view of the realities for current Internet mail, please say so.

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked], Dave Crocker <[email-masked]>
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Date: Tue, 21 Dec 2004 19:27:18 -0500
Cc: Tony Finch <[email-masked]>
Sender: [email-masked]

On Tue December 21 2004 19:02, Dave Crocker wrote:

> what's the problem with mere enforcement?  why must the msa create fields?

E.g. to add the mandatory Date header field if missing
(the semantics may be wrong (submission time rather than
creation time)) (the MUA might have no reliable clock
from which it could generate date-time).

From: Dave Crocker <[email-masked]>
To: Tony Finch <[email-masked]>
CC: <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Tue, 21 Dec 2004 16:02:15 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

> > saying anything that means the msa always creates something is a writing
> > error.  my intent is to say that it enforces things and *might*
> > create/modify things, in order to do the enforcing.
>
>  How can it enforce the envelope recipient list if the mesage arrived via
>  RFC 2476?

submission has an envelope, just like smtp, since they are essentially the same protocol.  so the question is what it means to 'enforce' existing fields:

1. there might be site constraints on the rfc2821.mailfrom values that a submitter can use.  (the fact the submission is expected to be an authenticated process makes enforcement of this feasible.)

2. Again, because submission is authenticated, there can be enforcement on rfc2822.from and other fields.

3. and the specific that you ask about, recipient list, is certainly not likely to need enforcement. but the recipient list wasn't singled out in the mail-arch doc.

what's the problem with mere enforcement?  why must the msa create fields?

>  I'm inclined to talk about message submission in the abstract, with some
>  functions typically implemented by the MUA (e.g. preparing the envelope)
>  and some by the MSA (header fix-ups and policy enforcement).

well, the intent was to permit the exact venue for some of these functions (mua vs. msa) to be fluid.)

>  "envelope":
>
> > User-to-User is the stuff that the originator creates with the intent
> > that it be consumed by the recipient.
>  >
> > In contrast, the handling information is created for consumption by the
> > handling service or by the handling service, primarily for consumption
> > by email operators or for operational analysis.
>
>  I conclude that there's no detailed, precise definition of which parts of
>  the message header might constitute "handling information", and might
>  therefore constitute part of the Crocker-envelope. The Finch-envelope, on
>  the other hand, is well-defined and commonly understood.

not by crocker (...)

I had not thought of my assessment on this as being particular to me, although i know there is variation within the community.  I do not have any sense of a solid consensus within the community.  If I did, I'd use it.

> > Good question.  I do not know what constitutes "prevailing" practise.
> > My own view is that old trace information should be dropped, since a
> > re-send is creating an entirely new message.
>
>  2822 implies preservation of all the trace fields (including previous
>  received and resent information).

sigh.  yeah.  it does.

>  "gateway"/"firewall":
>
> > >  We need a word to talk about the parts of the MHS that might mangle
> > >  messages.
>  >
> > Historically, the email term for an entity that transforms content is 'gateway'.
> > A gateway is an entity that does message (semantics) conversions.
>
>  I seem to remember discussing this with you in the past. I asked whether
>  MTAs that do content mangling were gateways and you said that they
>  probably don't count because they don't gateway into a non-internet MHS,
>  and that "MTA firewall" might be a better term.

well, i certainly could imagine having said that at some point.  the history of email is not filled with care or precision about functional boundaries.  these days i'd rather keep these things as simple as possible.  it's the only way we are ever going to a) get clear about things, and b) stand any chance of getting meaningful conformance.

>  I like the idea of distinguishing between gatewaying and (for want of a
>  better term) firewalling. The former seems to imply semantics-preserving
>  transformations (or the closest possible approximation thereto), but a
>  firewall might make all sorts of content-mangling or -destroying changes.

My own preference is to have a single manglers, and then try to define sub-categories.  So, a security gateway is a firewall.  a mail protocol gateway is something like x.400/internet.

> > Relative to the underlying handling service, when there is a gateway it
> > usually take formal delivery creates a new message.
>
>  Do you mean "final delivery" here in the 821 sense, or do you mean the
>  kind of responsibility hand-off that 1123 and 2821 describe? From the
>  point of view of "emulat[ing] a continuattion of the handling service" I
>  think it is better to view final delivery as an accident of implementation
>  rather than part of the architecture. 

Let me see if I can get away with making this simple:  final delivery is when a DNS success message would get generated...

Back when I was arguing that imap was post-delivery and pop effected delivery, folks were pointed is citing the DSN generation event as a discriminator.  So I am hoping it can help clarify this situation too.

>A gateway is more likely to be
>  implemented in a manner similar to a firewall-MTA than off the back of an
>  MDA.

depends.

> > As I said, what makes gateways interesting -- besides the impossible
> > task of trying to preserve all the semantics -- is that it is possible
> > that they emulate a continuattion of the handling service, through the
> > gateway.  This is difficult and I believe it is rare.
>
>  I understand that Exchange and Notes etc. work in essentially this way,
>  which goes a long way to explain the terrible mangling that they cause.

attempts to to application gatewaying that preserves interesting end-to-end properties is very tough.

>  dropping/replacement of attachments which violate some restriction which
>  might be security-related (e.g. no executables, virus infection) or might
>  be for some other policy reason (e.g. no HTML messages on a mailing list,
>  or no graphics, or whatever).

yeah.  i take your point.  

interesting modeling challenge.

>  adding footers, such as list unsubscription instructions or stupid legal
>  disclaimers or ads

well, that's not a firewall.  that's a mailing list and i model that as a separate user, creating a new message...

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: comments on draft-crocker-email-arch-01
Date: Tue, 21 Dec 2004 14:17:43 -0500
Sender: [email-masked]

FYI -- Dave sent this to the ASRG list earlier this year; it
may be useful for discussions of the architecture draft:

> Date: Wed, 18 Feb 2004 11:20:47 -0800
> From: Dave Crocker <[email-masked]>
> Reply-To: Dave Crocker <[email-masked]>
> Organization: Brandenburg InternetWorking
> To: [email-masked]
> Subject: [Asrg] Patrik Falstrom's mailflow chart
> 
> I don't remember seeing this mentioned, but think it worth highlighting
> for folks:
> 
> Patrik Falstrom did a mail flow control chart that is _extremely_
> detailed.  I've been doing a simplistic one, for easy discussion, but
> his is very thorough.
> 
> It is at <http://www.ripe.net/ripe/meetings/ripe-47/mailflows.pdf>

I've also put up a version of Patrik's chart at
http://users.erols.com/blilly/mailflow/mailflow.pdf
(the directory also contains "dot" source ("graphviz"
software) and a makefile)

Date: Tue, 21 Dec 2004 17:44:03 +0000
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked]
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1870869256-1358188998-1103650056=:24304
Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE
Content-ID: <[email-masked]>

> >=A0>=A0I definitely am used to thinking of protocols as residing at boun=
daries,
> >=A0>=A0pretty much by definition, and I hope we are VERY careful about
> >=A0>=A0maintaining the distinction between the abstract architecture, ve=
rsus
> >=A0>=A0concrete implementations.
> >
> >=A0 I agree. However in the ugly tables you say that the envelope is cre=
ated
> >=A0 by the MSA, but from a the protocols point of view (and from the poi=
nt of
> >=A0 view of most software these days) the MUA must create the envelope i=
n
> >=A0 order to be able to talk to the MSA.
>
> saying anything that means the msa always creates something is a writing
> error.  my intent is to say that it enforces things and *might*
> create/modify things, in order to do the enforcing.

How can it enforce the envelope recipient list if the mesage arrived via
RFC 2476?

I'm inclined to talk about message submission in the abstract, with some
functions typically implemented by the MUA (e.g. preparing the envelope)
and some by the MSA (header fix-ups and policy enforcement).

"envelope":

> User-to-User is the stuff that the originator creates with the intent
> that it be consumed by the recipient.
>
> In contrast, the handling information is created for consumption by the
> handling service or by the handling service, primarily for consumption
> by email operators or for operational analysis.

I conclude that there's no detailed, precise definition of which parts of
the message header might constitute "handling information", and might
therefore constitute part of the Crocker-envelope. The Finch-envelope, on
the other hand, is well-defined and commonly understood.

>> When a message is Resent- are the trace=A0fields from the original
>> message now part of the "user-to-user message" or not?
>
> Good question.  I do not know what constitutes "prevailing" practise.
> My own view is that old trace information should be dropped, since a
> re-send is creating an entirely new message.

2822 implies preservation of all the trace fields (including previous
received and resent information). Most implementations, however, follow
822 which only allows for one level of resending. It's common for them to
preserve the old trace fields except that they are prefixed with X-.

"gateway"/"firewall":

> >=A0 We need a word to talk about the parts of the MHS that might mangle
> >=A0 messages.
>
> Historically, the email term for an entity that transforms content is 'ga=
teway'.
> A gateway is an entity that does message (semantics) conversions.

I seem to remember discussing this with you in the past. I asked whether
MTAs that do content mangling were gateways and you said that they
probably don't count because they don't gateway into a non-internet MHS,
and that "MTA firewall" might be a better term.

I like the idea of distinguishing between gatewaying and (for want of a
better term) firewalling. The former seems to imply semantics-preserving
transformations (or the closest possible approximation thereto), but a
firewall might make all sorts of content-mangling or -destroying changes.

> Relative to the underlying handling service, when there is a gateway it
> usually take formal delivery creates a new message.

Do you mean "final delivery" here in the 821 sense, or do you mean the
kind of responsibility hand-off that 1123 and 2821 describe? From the
point of view of "emulat[ing] a continuattion of the handling service" I
think it is better to view final delivery as an accident of implementation
rather than part of the architecture. A gateway is more likely to be
implemented in a manner similar to a firewall-MTA than off the back of an
MDA.

> As I said, what makes gateways interesting -- besides the impossible
> task of trying to preserve all the semantics -- is that it is possible
> that they emulate a continuattion of the handling service, through the
> gateway.  This is difficult and I believe it is rare.

I understand that Exchange and Notes etc. work in essentially this way,
which goes a long way to explain the terrible mangling that they cause.

> I am not used to having the term firewall refer to the job of content
> conversion.  I am used to it strictly meaning a security function of
> detecting attacks and rejecting them.

Yes, I agree that the term doesn't fit very well. The functions I want to
describe with the word are mostly security-related but some of them are
more or less technical policy.

dropping/replacement of attachments which violate some restriction which
might be security-related (e.g. no executables, virus infection) or might
be for some other policy reason (e.g. no HTML messages on a mailing list,
or no graphics, or whatever).

adding footers, such as list unsubscription instructions or stupid legal
disclaimers or ads

Tony.
--=20
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOLE LUNDY FASTNET: NORTHWEST 4 BACKING WEST OR SOUTHWEST 5 TO 7, PERHAPS
GALE 8 LATER. RAIN LATER. GOOD BECOMING MODERATE.
--1870869256-1358188998-1103650056=:24304--

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked]
Subject: Re: The concepts of email address and mailbox
Date: Tue, 21 Dec 2004 08:02:22 -0500
Cc: Jochen Topf <[email-masked]>
Sender: [email-masked]

On Sun December 19 2004 06:27, Jochen Topf wrote:
> 
> Hi!
> 
> Reading Dave Crocker's draft-crocker-email-arch-01 I started to think again
> about the identification and differences between "email addresses" and
> "mailboxes". In the draft, as in many other documents, both concepts are
> essentially one, the email address is a "mailbox address".

The essential difference between "address" and "mailbox" is
that an "address" can be either a "mailbox" or a named group.
RFC 822:

     6.1.  SYNTAX

     address     =  mailbox                      ; one addressee
                 /  group                        ; named list

     group       =  phrase ":" [#mailbox] ";"

     mailbox     =  addr-spec                    ; simple address
                 /  phrase route-addr            ; name & addr-spec

RFC 2822:

address         =       mailbox / group

mailbox         =       name-addr / addr-spec

name-addr       =       [display-name] angle-addr

angle-addr      =       [CFWS] "<" addr-spec ">" [CFWS] / obs-angle-addr

group           =       display-name ":" [mailbox-list / CFWS] ";"
                        [CFWS]

> Historically email is send to a mailbox

Or as stated in both 822 and 2822 "A mailbox receives mail".

> [...] There is a one-to-one relationship between 
> addresses and the mailboxes.

No! An address may contain many mailboxes.

> Mailboxes were usually associated with 
> operating system users

Not necessarily. 822/2822 discussion of mailbox continues
with: "It is a  conceptual  entity  which does not necessarily
pertain to file storage.  For example, some sites may choose to
print mail on their line printer and  deliver the output to the
addressee's desk."  That text also appeared in RFCs 733 and 724.

> , so the local_part is really the username. So 
> far so good.

No, you've gone wrong w.r.t. "address" vs. "mailbox", and
in associating "mailbox" with "operating system user". After
that, things get more confused. Please don't try to redefine
standard terms that have been in use for more than a quarter
century.

From: Dave Crocker <[email-masked]>
To: Tony Finch <[email-masked]>
CC: <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Tue, 21 Dec 2004 08:56:29 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

> > legacy term.  orig/recip are from the X.400 days and have a pretty high
> > installed base of usage.  I do not see our community having a dominant
> > practise of using author -- although it does seem to have a high enough
> > occurrence to cite the term also -- and this document is not trying to
> > establissh new practise.
>
>  Hmm. My problem is that "originator" seems ambiguous between "author(s)"
>  (i.e. 822.From) and "submitter" (822.Sender).

understand.

however, given the extensive of using the term originator, i'm inclined to change it only if there is a groundswell of popular preference expressed.  

again, note that the goal for the document is to document, not really be creative or even persuasive.  it is trying to capture the current community consensus on existing email architecture (and terminology).

> > I definitely am used to thinking of protocols as residing at boundaries,
> > pretty much by definition, and I hope we are VERY careful about
> > maintaining the distinction between the abstract architecture, versus
> > concrete implementations.
>
>  I agree. However in the ugly tables you say that the envelope is created
>  by the MSA, but from a the protocols point of view (and from the point of
>  view of most software these days) the MUA must create the envelope in
>  order to be able to talk to the MSA. 

saying anything that means the msa always creates something is a writing error.  my intent is to say that it enforces things and *might* create/modify things, in order to do the enforcing.

> > > MDA:
>
> > The only question this leaves is about choices for delivery protocols.
> > I think the answer is reduced to <local> or SMTP.
>
>  LMTP is intended for this role.

The document is trying to capture actual practise.  That's why this particular diagram shows lists of candiates.  It is my impression that LMTP is not used much.

Also, on review, the MDA-MS, MS-MS and MS-MUA segments can have quite a bit of overlap in the set of protocols they use.  In particular, POP and IMAP do get used in each of the segments, I believe.

> > >  Sieve should be mentioned as a means for controlling the MDA.
>  >
> > The diagram shows sieve as feeding into the MDA.
>
>  Hmm. Sieve's a scripting language. What architectural entity interprets
>  it?

The diagram shows the sieve data being given to the MDA.  The intent is to indicate that the MDA does the interpreting.

> > >  Is it worth distinguishing online mode and local message store access?
>  >
> > although the operational difference is important, how does that affect
> > the architecture?
>
>  Whether an Internet protocol is being used or not.

Since this is a document about Internet mail, references to non-Internet stuff is intentionally limited.

What, specific change are you suggesting?

Note that the diagram already shows two message stores.  One of the reasons is that the second one can be used offline.

>  The details of communication with the message store and where it resides
>  etc. in this section are rather complicated, because a lot of the
>  functions can be implemented in either or both of two places (on the
>  server and/or the client), with or without network protocols in between.

right.

> > >  Sieve should be mentioned as a means for controlling the movement of
> > >  messages in offline mode under control of the rMUA.
>  >
> > please elaborate.
>
>  In the IMAP world, the MDA interprets a Sieve script to determine which
>  mailboxes to deliver the message into, and the MUA talks IMAP to the
>  message store to access the email.

I think the problem is that the document shows sieve in the diagram but does not discuss it.  I need to add some text, along the lines of what you are saying.

>  I think it's essential to have a word to describe the SMTP commands that
>  occur before DATA (i.e. MAIL FROM and RCPT TO). See above in the
>  discussion about MSA for an example. I'm not sure what you mean by
>  "user-to-user message" and how it can be used to draw a line through the
>  message header. 

This is the major architectural weakness to Internet mail.  We blurred the distinction between user headers and handling headers.  

User-to-User is the stuff that the originator creates with the intent that it be consumed by the recipient.  

In contrast, the handling information is created for consumption by the handling service or by the handling service, primarily for consumption by email operators or for operational analysis.

>The Subject: is probably included. 

yes.

>Are the addressing
>  fields included? 

Ignoring the MUA/MSA protocol issues, where the MSA might get its rcptto list from the to/cc/bcc fields, the to/cc/bcc fields are intended for consumption by recipients.

>What about instructions to the MSA (BCC:)? 

depends upon the particulars of the mua/msa interface.

>What about
>  Date:? 

It really is not used by the handling system and very much IS used by recipients.

>Message-ID:? 

mixed bag, on this one, i think.

>X-Mailer:? 

operations use.

> When a message is Resent- are the trace
>  fields from the original message now part of the "user-to-user message" or
>  not?

Good question.  I do not know what constitutes "prevailing" practise.  My own view is that old trace information should be dropped, since a re-send is creating an entirely new message.

> > >  There should also be a section on MTA firewalling, to distinguish it
> > >  from gatewaying.
>  >
> > I'm inclined to agree, but let's explore this a bit.  What is the
> > architectural role of a firewall?  I understand it's operational import,
> > of course, but how does this relate to an/the architecture?
>
>  We need a word to talk about the parts of the MHS that might mangle
>  messages. You mentioned fleshing out the operational side of the
>  architecture, so that we can talk about outgoing relays and incoming MXs
>  etc. and in this context the border firewall MTA is becoming practically
>  universal.

Historically, the email term for an entity that transforms content is 'gateway'.

A gateway is an entity that does message (semantics) conversions.

> > >  MUA Gateways:
> > >
> > >  Aren't these really at the MTA level? Putting them at the MUA level
> > >  implies a "final delivery" event before the message reaches the
> > >  gateway.

That's right.  A gateway sits between heterogeneous environments and does conversions to reconcile differences.  By contrast, a relay makes no changes to message semantics and merely takes the message off one transfer link, makes a routing decision, and queues the message for another transfer link.

Relative to the underlying handling service, when there is a gateway it usually take formal delivery creates a new message.  As I said, what makes gateways interesting -- besides the impossible task of trying to preserve all the semantics -- is that it is possible that they emulate a continuation of the handling service, through the gateway.  This is difficult and I believe it is rare.

> > Gateways are strange beasts, indeed.  My experience forces me to view
> > them as hybrid MTA/MDA/MUA devices.  The basic point is that a gateway
> > messes mightily with the content.  So it is not merely a variant of an
> > MTA.  On the other hand, your point about the possibility of deferring
> > the formal delivery event to a point after the gateway is well taken.
>
>   From the content mangling point of view they are close to firewall MTAs.
>  They are clearly inside the MHS boundary so are not MUAs.

I am not used to having the term firewall refer to the job of content conversion.  I am used to it strictly meaning a security function of detecting attacks and rejecting them.

That a firewall might do the 'rejecting' by selectively modifying the content is certainly true, but it's secondary to the task that is referenced by the term firewall.

My point is not to be abstractly pure or to deny the reality of what is done in existing practise. Rather I want to make sure the architecture is as clean as possible in distinguishing roles and responsibilities.  In this regard, one thing we keep needing to juggle is the difference between the conceptual architecture versus implementation choices that might cross architectural boundaries (and/or blur them).

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

Date: Tue, 21 Dec 2004 12:41:15 +0000
From: Tony Finch <[email-masked]>
To: Dave Crocker <[email-masked]>
cc: [email-masked]
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--1870869256-1265857937-1103632218=:19954
Content-Type: TEXT/PLAIN; CHARSET=ISO-8859-1
Content-Transfer-Encoding: QUOTED-PRINTABLE
Content-ID: <[email-masked]>

Most of your responses seem to be in general agreement, which is good.
Some follow-up comments:

> >=A0 Originator/Recipient: Why not use the term "Author" instead of
> >=A0 "Originator"?
>
> legacy term.  orig/recip are from the X.400 days and have a pretty high
> installed base of usage.  I do not see our community having a dominant
> practise of using author -- although it does seem to have a high enough
> occurrence to cite the term also -- and this document is not trying to
> establissh new practise.

Hmm. My problem is that "originator" seems ambiguous between "author(s)"
(i.e. 822.From) and "submitter" (822.Sender).

> >=A0 MSA:
> >
> >=A0 It should be noted that the submission protocol does not lie on
> >=A0 the boundary between the MSA and MUA, because when it is being used
> >=A0 the user's software has to do some MSA work like setting up the
> >=A0 message envelope and handling the BCC: header field. An example of
> >=A0 an implementation boundary which lies on the architectural boundary =
is
> >=A0 the `sendmail -t` API in Unix.
>
> I definitely am used to thinking of protocols as residing at boundaries,
> pretty much by definition, and I hope we are VERY careful about
> maintaining the distinction between the abstract architecture, versus
> concrete implementations.

I agree. However in the ugly tables you say that the envelope is created
by the MSA, but from a the protocols point of view (and from the point of
view of most software these days) the MUA must create the envelope in
order to be able to talk to the MSA. Hence my chatter about `sendmail -t`
being the canonical MSA interface, rather than RFC 2476.

> >  This relates to
> >=A0 whether the MSA can be said to "set" a HELO identity (I would say
> >=A0 not). Additional confusion is caused by the submission protocol, in
> >=A0 which a HELO identity is "set" by the MUA part of the MSA.
>
> Whatever entity is doing the client side of smtp (or submit) is the
> entity that sets the helo identity.

Again, inconsistency in the tables. The tables say "submitter" but should
say "operator" or "administrator" (which is a role we don't have at the
moment).

> > MDA:

> The only question this leaves is about choices for delivery protocols.
> I think the answer is reduced to <local> or SMTP.

LMTP is intended for this role.

> >=A0 Sieve should be mentioned as a means for controlling the MDA.
>
> The diagram shows sieve as feeding into the MDA.

Hmm. Sieve's a scripting language. What architectural entity interprets
it?

> >=A0 Is it worth distinguishing online mode and local message store acces=
s?
>
> although the operational difference is important, how does that affect
> the architecture?

Whether an Internet protocol is being used or not.

The details of communication with the message store and where it resides
etc. in this section are rather complicated, because a lot of the
functions can be implemented in either or both of two places (on the
server and/or the client), with or without network protocols in between.

> >=A0 Sieve should be mentioned as a means for controlling the movement of
> >=A0 messages in offline mode under control of the rMUA.
>
> please elaborate.

In the IMAP world, the MDA interprets a Sieve script to determine which
mailboxes to deliver the message into, and the MUA talks IMAP to the
message store to access the email.

In the POP world, the MDA delivers into a single mailbox per user, and the
MUA periodically copies it to a local store. In the course of storing the
email locally the MUA can interpret a Sieve script to determine which
local mailbox to store each message in.

> >=A0 Message Data:
> >
> >=A0 I find it useful to distinguish between the "message envelope" (i.e.
> >=A0 the sequence of SMTP commands before DATA) and the "message data"
> >=A0 (which consists of everything after DATA, i.e. the "message header"
> >=A0 and the "message body"). Before this document I haven't heard of
> >=A0 anyone including trace fields in the envelope.
>
> heh. heh.  I think I'm not the only one that has been doing it for many
> years. But, yes, I know it is not the view of lots of folks.  It's
> probably worth having some discussion about this.
>
> The reality of the trace fields is that they are strictly the result of
> the handling service.  They have nothing to do with the user-to-user
> message.  That's the basis for drawing the line, in my view.  The fact
> that some handling information is passed in smtp commands and others are
> passed in rfc2822 headers is secondary, in my view.
>
> comments?

I think it's essential to have a word to describe the SMTP commands that
occur before DATA (i.e. MAIL FROM and RCPT TO). See above in the
discussion about MSA for an example. I'm not sure what you mean by
"user-to-user message" and how it can be used to draw a line through the
message header. The Subject: is probably included. Are the addressing
fields included? What about instructions to the MSA (BCC:)? What about
Date:? Message-ID:? X-Mailer:? When a message is Resent- are the trace
fields from the original message now part of the "user-to-user message" or
not?

> >=A0 MTA Relaying:
> >
> >=A0 There should also be a section on MTA firewalling, to distinguish it
> >=A0 from gatewaying.
>
> I'm inclined to agree, but let's explore this a bit.  What is the
> architectural role of a firewall?  I understand it's operational import,
> of course, but how does this relate to an/the architecture?

We need a word to talk about the parts of the MHS that might mangle
messages. You mentioned fleshing out the operational side of the
architecture, so that we can talk about outgoing relays and incoming MXs
etc. and in this context the border firewall MTA is becoming practically
universal.

> >=A0 MUA Gateways:
> >
> >=A0 Aren't these really at the MTA level? Putting them at the MUA level
> >=A0 implies a "final delivery" event before the message reaches the
> >=A0 gateway.
>
> Gateways are strange beasts, indeed.  My experience forces me to view
> them as hybrid MTA/MDA/MUA devices.  The basic point is that a gateway
> messes mightily with the content.  So it is not merely a variant of an
> MTA.  On the other hand, your point about the possibility of deferring
> the formal delivery event to a point after the gateway is well taken.

From=20the content mangling point of view they are close to firewall MTAs.
They are clearly inside the MHS boundary so are not MUAs.

> >=A0 MUA Mailing Lists:
>
> Mailing lists are one example of a class of "user-level multi-hop
> process" functions that are part of group activity, including such fun
> stuff as organizational flow mechanisms like purchase approval".  I
> think the architectural reference needs to encompass this range of
> functions, rather than being tailored to mailing lists.

That sounds like a useful way of clarifying things.

Tony.
--=20
f.a.n.finch  <[email-masked]>  http://dotat.at/
SOLE LUNDY FASTNET: NORTH BACKING WEST OR SOUTHWEST 5 OR 6, INCREASING 7
PERHAPS GALE 8 LATER. RAIN LATER. GOOD BECOMING MODERATE OCCASIONALLY POOR.
--1870869256-1265857937-1103632218=:19954--

Date: Tue, 21 Dec 2004 11:55:21 +0000
From: Tony Finch <[email-masked]>
To: Jochen Topf <[email-masked]>
cc: [email-masked]
Subject: Re: The concepts of email address and mailbox
Sender: [email-masked]

On Sun, 19 Dec 2004, Jochen Topf wrote:
>
> Reading Dave Crocker's draft-crocker-email-arch-01 I started to think again
> about the identification and differences between "email addresses" and
> "mailboxes". In the draft, as in many other documents, both concepts are
> essentially one, the email address is a "mailbox address".
>
> I recognize the historical truth in this and the current email systems
> have many vestiges of that school of thought, but nevertheless, I think it
> is conceptually misleading. I am not sure where this discussion will lead
> and whether this is important for the draft on email architecture, but
> I'll send it to the list anyway and we'll see where it goes.

I agree with everything you say.

I have a very minor niggle, though.

> It is the job of the MDA to do the actual transition between the "email
> address space" and the "mailbox space". It translates email addresses
> into mailbox names and delivers the mail.

> Note that in this concept an forward/alias from one email address to another
> happens in the MTA not in the MDA.

It's often the case that it isn't until the email address -> mailbox
translation stage in the MDA that the system finds out that the
destination is another email address instead of a mailbox. For example,
Sieve implementations can direct messages to mailboxes or email addresses.

MTAs can also to email address -> email address routing. I'm not sure if
this is an implementation artefact or an architectural feature. It's often
hard to distinguish between the MDA and MTA parts of email software.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
PORTLAND PLYMOUTH: NORTH OR NORTHWEST 4 OR 5, BUT 6 AT FIRST IN PLYMOUTH,
BACKING WEST 5 OR 6 LATER. SHOWERS THEN RAIN. GOOD BECOMING MODERATE OR POOR.

From: Dave Crocker <[email-masked]>
To: <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Tue, 21 Dec 2004 00:06:25 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

On Mon, 20 Dec 2004 14:01:30 -0500, Bruce Lilly wrote:
>     published? It seems to be quite similar; at least a few
>     items previously commented on appear to be unchanged.

well, this is pretty amazing.  my software did something weird to your message and a couple of others.

sorry to ask this yet again, but please send your message yet again.

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Dave Crocker <[email-masked]>
To: <[email-masked]>, <[email-masked]>, Dave Crocker <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Mon, 20 Dec 2004 11:53:10 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

On Mon, 20 Dec 2004 14:01:30 -0500, Bruce Lilly wrote:
>  On Sat December 18 2004 15:56, Dave Crocker wrote:
>
> >         An html copy of the latest (unreleased) draft is at:
> >
> > <http://brandenburg.com/specifications/draft-crocker-mail-arch-01.html>
>
>  1. Is that supposed to be different in content from the I-D
>     published? It seems to be quite similar; at least a few
>     items previously commented on appear to be unchanged.

it is meant to have changes from earlier comments, but yes, it looks like i missed quite a few.  please re-send them to me privately, since my archive is messed up.

>  2. If different, is there a text version available (HTML
>     is a PITA, particularly when offline).

<http://brandenburg.com/specifications/draft-crocker-mail-arch-01.txt>

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Dave Crocker <[email-masked]>
To: Bruce Lilly <[email-masked]>, <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Mon, 20 Dec 2004 11:52:31 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

On Mon, 20 Dec 2004 14:01:30 -0500, Bruce Lilly wrote:
>  On Sat December 18 2004 15:56, Dave Crocker wrote:
>
> >         An html copy of the latest (unreleased) draft is at:
> >
> > <http://brandenburg.com/specifications/draft-crocker-mail-arch-01.html>
>
>  1. Is that supposed to be different in content from the I-D
>     published? It seems to be quite similar; at least a few
>     items previously commented on appear to be unchanged.

it is meant to have changes from earlier comments, but yes, it looks like i missed quite a few.  please re-send them to me privately, since my archive is messed up.

>  2. If different, is there a text version available (HTML
>     is a PITA, particularly when offline).

<http://brandenburg.com/specifications/draft-crocker-mail-arch-01.txt>

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: [email-masked], Dave Crocker <[email-masked]>
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Date: Mon, 20 Dec 2004 14:01:30 -0500
Sender: [email-masked]

On Sat December 18 2004 15:56, Dave Crocker wrote:

> 	An html copy of the latest (unreleased) draft is at:  
> 
> <http://brandenburg.com/specifications/draft-crocker-mail-arch-01.html>

1. Is that supposed to be different in content from the I-D
   published? It seems to be quite similar; at least a few
   items previously commented on appear to be unchanged.

2. If different, is there a text version available (HTML
   is a PITA, particularly when offline).

Date: Sun, 19 Dec 2004 12:27:33 +0100
From: Jochen Topf <[email-masked]>
To: [email-masked]
Subject: The concepts of email address and mailbox
Sender: [email-masked]

Hi!

Reading Dave Crocker's draft-crocker-email-arch-01 I started to think again
about the identification and differences between "email addresses" and
"mailboxes". In the draft, as in many other documents, both concepts are
essentially one, the email address is a "mailbox address".

I recognize the historical truth in this and the current email systems
have many vestiges of that school of thought, but nevertheless, I think it
is conceptually misleading. I am not sure where this discussion will lead
and whether this is important for the draft on email architecture, but
I'll send it to the list anyway and we'll see where it goes.

Historically email is send to a mailbox, either on the current machine
(so you only need the local_part) or to another host (then you have the
local_part@host address). There is a one-to-one relationship between
addresses and the mailboxes. Mailboxes were usually associated with
operating system users, so the local_part is really the username. So
far so good.

But already with the introduction of forwarding this changed a little bit.
If you have a .forward, mail for a user can be redirected somewhere else.
The distinct identity of the user is still here. His mailbox will always
be empty, but he has the .forward file. But once you move the forwarding
to the /etc/aliases file, you can create email addresses that belong to
no real mailbox and to no real entity at all.

So instead of thinking of email addresses and mailboxes as the same thing,
we should think of them as different entities. An "email address" is
something I can send an email to, no more, no less. A "mailbox" is
something used to store email, it can be associated with one or more
addresses, but it doesn't have to be.

Lets look at a few examples of email use with the two different concepts
in mind:

1. The outbox

Many MUAs store all outgoing email in an "outbox". Its a mailbox without
an address.

2. Sub mailboxes

With IMAP you can have "sub-mailboxes". They all have the same address,
but they also have a distinct name.

3. Addresses without mailboxes

There are many email addresses to which mail can be send that will never
turn up in a mailbox. Fax forwards come to mind or other systems where
the contents of the arriving email are used to start some kind of process.
Mailing lists always send the mail on, there is no mailbox involved (there
might be some kind of archive though).

4. Splitting email

Email to one address can be forwarded/aliased to several addresses, say
your home system and your laptop. Essentially email to one address gets
delivered to two mailboxes. With most current systems we have to give
those mailboxes distinct email addresses to allow this. But those email
addresses are never used outside your system, just internal ways of
naming. Wouldn't it be conceptually simpler to have a distinct namespace
for email addresses (which is global) and one for mailboxes (which is
internal to your organisation).

If you do the splitting before the mailbox, this would also get rid of
the ugly recursion problem in .forward files, where you put the name
of the mailbox itself in the .forward file to mean that you want an
email also delivered to the mailbox itself.

Concept 1:

  [email-masked] ---> foo -+-> \foo
                            |
                            +-> [email-masked]

Conecpt 2:

  ADDR/[email-masked] -+-> MAILBOX/foo 
                        |
		        +-> ADDR/[email-masked]

(With the ADDR/ and MAILBOX/ I want to denote that there are two different
namespaces involved.)

5. Two addresses into one mailbox

If you have two email addresses and want email delivered into the same
mailbox you essentially have to have one mailbox and one forward:

  [email-masked] -> mb
  [email-masked] -> [email-masked]

When you delete the foo address, you get a dangling link. But if you
have distinct email addresses and mailboxes this keeps working:

  ADDR/[email-masked] -+
                        +-> MAILBOX/foo
  ADDR/[email-masked] -+

Of course this only works inside one organisation.

5. Automatic classification of email

The spam problem has brought the automatic classification of email to
the forefront. Many people have not one but two mailboxes behind every
email address, the normal inbox and the spam box. Others have even more
sophisticated classification for different parts of there email. Work email
here, private email there, mailing lists traffic over there. So the
email address alone is not the only thing deciding into which mailbox
email gets delivered, there are many other attributes.

6. Virtual domains

The modern usage of bazillions of domains has led to mail systems which
are responsible for many domains at once. And of course the local parts
aren't distinct between all those domains. The old way of mapping the
local part to the name of a user or a mailbox doesn't work any more.
(and the way this is often done by mapping all virtual domains to one
real domain is a hack).

In many simple cases an email address and a mailbox will still be the
same and can be accessed under the same name. I am only concerned here
with the conceptual model not with the actual names used and the way the
software works.

The whole discussion might be a bit academic, altough I have build large
email systems around this concept which uses mailbox names totally distinct
from email addresses etc. and it works in real life, too.

But, more important, I think it might shed some light on the distribution
of work between the components of a mail system, i.e. the MTA, MDA and MUA.
In my view the MUA knows about email addresses only to send email. To read
email it knows about mailboxes (and, usually, the hosts where they reside).
It does not have to know about the email address used to send email to
those mailboxes. (And it doesn't actually know, because it doesn't see
the RFC2821.To address.) Similarly the POP and IMAP servers don't know
about email addresses, only about mailboxes.

It is the job of the MDA to do the actual transition between the "email
address space" and the "mailbox space". It translates email addresses
into mailbox names and delivers the mail.

To make this point more clear: There are two namespaces involved in email
today: One is the "email address" namespace. It consists of a domain,
resolved through DNS and the use of MX records and a locally unique
local part which should be transparent for the rest of the world.
Then there is the totally distinct "mailbox" namespace which consists
of a protocol (POP or IMAP), a hostname and a mailbox name. (And of course,
there are local mailboxes which is just a special case of a "local protocol".)

Delivery doesn't have to go into a mailbox. A fax forward for instance means
that the MDA will decide that a specific email address should be delivered
"to a telephone number" and will do this. But, again, there is this transition
between email address and another namespace. (By the way: using addresses
like [email-masked] for fax delivery is horrible. Addresses
like this mix up an already perfect and globally unique address, i.e. the
fax number, with the delivery mechanism through the gateway at
phone.example.com. Users don't want to and shouldn't have to know about the
@phone.example.com part, after all we got rid of bang paths a long time
ago.)

Note that in this concept an forward/alias from one email address to another
happens in the MTA not in the MDA.

Jochen
-- 
Jochen Topf  [email-masked]  http://www.remote.org/jochen/  +49-721-388298

From: Dave Crocker <[email-masked]>
To: <[email-masked]>
Reply-To: Dave Crocker <[email-masked]>
Date: Sat, 18 Dec 2004 12:56:21 -0800
Subject: Re: [long] commnets on draft-crocker-email-arch-01
Sender: [email-masked]

Tony,

Sorry it's taken so long for me to respond.  Many thanks for your thoughtful comments.  

FOLKS -- 

	I'm doing another pass over the document.  I'm hoping this is the last before seeking to publish it, preferably with some IETF status, not just as an Informational RFC.  So, please think about making comments.

	An html copy of the latest (unreleased) draft is at:  

<http://brandenburg.com/specifications/draft-crocker-mail-arch-01.html>

>  Roles:
>  I'm not sure about the "Provider" concept. This is partly because I
>  don't like the term (it's too generic), partly because I'm not sure
>  it's useful - it isn't used elsewhere in the document, though the term
>  "organization" is.

I do not know what to add, about them, in an architecture document.  Still, providers have become a pretty distinct focus of attention for the anti-spam discussions, so I think they need to be cited in this sort of document.  

I'll take them out of the diagram, because it makes things confusing, but will leave a descriptive paragraph in.

>  Originator/Recipient: Why not use the term "Author" instead of
>  "Originator"?

legacy term.  orig/recip are from the X.400 days and have a pretty high installed base of usage.  I do not see our community having a dominant practise of using author -- although it does seem to have a high enough occurrence to cite the term also -- and this document is not trying to establish new practise.

> It should be noted that a message may have multiple
>  authors and recipients. 

yeah.

>  Relay: Is it worth breaking down this category further, ...
> These distinctions are
>  useful for talking about different kinds of access control

hmm.  maybe the way to resolve this and your concern over the reference to providers is to strengthen the distinction between the functional architecture (which is what this section is really about) from the 'operational' architecture.  So I'm inclined to enhance that latter section, as you cite, making the construct of an operational architecture distinct and significant.

>  Mailbox addresses:
>
>  Is it worth adding examples?

always.

>  Message IDs:
>
>  It might be worth explicitly mentioning that messages with the same
>  message ID can be assumed to be the same, e.g. for the purposes of
>  reducing the amount of space required by a message store.

and detecting duplicates.

>  I'm not sure what the following sentence refers to: "Although Internet
>  mail standards provide for a single identifier, more than one is
>  sometimes assigned."

As a message travels along its path, different handlers can add  additional message identifiers.

>  Identity Reference Convention:
>
>  Why MailFrom but Rcpt-To?

"A foolish consistency is the hobgoblin of little minds"

(which is my way of trying to dodge the reality that I'm lousy at being consistent.)

I'm inclined to prefer no dash, since folks might think the dash is required.

>  Email System Architecture:
>
>  The overview diagram is very unclear, 

yeah.  i need to rework it a bit.

>  Should there be a concept of privilege? I.e. MUAs are not privileged,
>  but everything else is. This is relevant to MSA fix-ups and alias
>  handling.

I'm going to add another diagram at the beginning, distinguishing users from the Mail Handling Service (another x.400 term).  I think that's the place to introduce the distinction about privilege.

It is also the place to make clear that some users do varous forms of posting of new messages that derive from mail they have received...

>  The comment about the BCC field is incorrect according to my
>  experience: I think it's more usual for the message to be submitted
>  once, and the MSA deletes the BCC header and transfers its address
>  list to the RCPT TO list when constructing the envelope.

actual practise varies, but certainly this scenario needs to be cited.

>  MSA:
>
>  It should be noted that the submission protocol does not lie on
>  the boundary between the MSA and MUA, because when it is being used
>  the user's software has to do some MSA work like setting up the
>  message envelope and handling the BCC: header field. An example of
>  an implementation boundary which lies on the architectural boundary is
>  the `sendmail -t` API in Unix.

I definitely am used to thinking of protocols as residing at boundaries, pretty much by definition, and I hope we are VERY careful about maintaining the distinction between the abstract architecture, versus concrete implementations.

That said, I think there is a philosophical debate one can have about where to draw lines on top of any functional series of 3 components (with the middle one being the protocol service). 

>  The MSA section should talk about the header fix-up work it should do,
>  including adding missing fields (Date:, Message-ID:) and correcting
>  the Sender: field (in the case that the MSA is more trusted than the
>  oMUA and the MSA has authenticated the Submitter).

mumble.  i think it makes sense to have the formal architecture document say that the msa must enforce certain requirements.  as for any actual changes it makes, i think of that as a matter of private arrangement between the mua and the msa, since 'enforcement' can range from creating the fields, to fixing the fields, to rejecting them if the are not present and correct.

>  Is the outgoing interface of the MSA required to be SMTP or can it be
>  a local hook into the first MTA? For example, in the case of `sendmail
>  -t` should the whole of sendmail be considered to be the MSA or is it
>  a Siamese twin of MSA and MTA (I prefer the latter)?

If this is a pure Internet Mail architecture, then I think it must be submit or smtp coming into the msa and smtp going out.

that a particular bit of software can operate as either/both an msa or mta is probably not relevant to the architecture...

> This relates to
>  whether the MSA can be said to "set" a HELO identity (I would say
>  not). Additional confusion is caused by the submission protocol, in
>  which a HELO identity is "set" by the MUA part of the MSA.

Whatever entity is doing the client side of smtp (or submit) is the entity that sets the helo identity.

>  One of the difficulties in the tables is that the HELO and Received:
>  point in different directions, by which I mean that the Received:
>  field refers to the message's previous hop whereas the HELO argument is
>  related to its next hop. 

The "BY" part of Received refers to the current hop, not the preceding one.  For simple relaying the HELO argument refers to the same operational entity, although of course it might choose to use a different identifier.

>  I also don't like referring to Received: as an identity because it is
>  really a composite field containing a number of identities - at least
>  the previous MTA's HELO, IP address etc. and this MTA's host name etc.
>  It's too complicated to fit into the model comfortably.

fair point.  it suggests distinguishes identities from the fields they occur in.

>  MDA:
>
>  Nothing in this table should be there!
>
>  POP and IMAP are not delivery protocols, 

This is extremely irritating.  I thought I had fixed this section and now I have no idea what is going on.  Sorry for the confusion.

For the original version of the document, I had a dogfight with folks about this point.  There was strong consensus against my view.  Hence I meant to change this to show pop and imap strictly as retrieval, as you suggest.

For the original version of the document, I had a dogfight with folks about this point.  There was strong consensus against my view.  Hence I meant to change this to show pop and imap strictly as retrieval, as you suggest.

The only question this leaves is about choices for delivery protocols.  I think the answer is reduced to <local> or SMTP.

>  Sieve should be mentioned as a means for controlling the MDA.

The diagram shows sieve as feeding into the MDA.

>  Is it worth distinguishing online mode and local message store access?

although the operational difference is important, how does that affect the architecture?

>  Sieve should be mentioned as a means for controlling the movement of
>  messages in offline mode under control of the rMUA.

please elaborate.

>  Message Data:
>
>  I find it useful to distinguish between the "message envelope" (i.e.
>  the sequence of SMTP commands before DATA) and the "message data"
>  (which consists of everything after DATA, i.e. the "message header"
>  and the "message body"). Before this document I haven't heard of
>  anyone including trace fields in the envelope.

heh. heh.  I think I'm not the only one that has been doing it for many years.

But, yes, I know it is not the view of lots of folks.  It's probably worth having some discussion about this.

The reality of the trace fields is that they are strictly the result of the handling service.  They have nothing to do with the user-to-user message.  That's the basis for drawing the line, in my view.  The fact that some handling information is passed in smtp commands and others are passed in rfc2822 headers is secondary, in my view.

comments?

>  Two Levels of Store-And-Forward:
>
>  Care is needed with the term "re-submission". It seems to imply that
>  RFC 2476 fix-ups occur, which is not the case in alias handling. I
>  still think alias handling is privileged, and architecturally an MDA
>  function rather than an MUA function: 

my own thinking has been undergoing some changes, given the recent discussions involving anti-spam work.  i'm inclined to agree with you.

what do other folks think?

>  MTA Relaying:
>
>  There should also be a section on MTA firewalling, to distinguish it
>  from gatewaying.

I'm inclined to agree, but let's explore this a bit.  What is the architectural role of a firewall?  I understand it's operational import, of course, but how does this relate to an/the architecture?

>  MUA Gateways:
>
>  Aren't these really at the MTA level? Putting them at the MUA level
>  implies a "final delivery" event before the message reaches the
>  gateway.

Gateways are strange beasts, indeed.  My experience forces me to view them as hybrid MTA/MDA/MUA devices.  The basic point is that a gateway messes mightily with the content.  So it is not merely a variant of an MTA.  On the other hand, your  point about the possibility of deferring the formal delivery event to a point after the gateway is well taken.

I'm not sure how to model this.  The simplest thing I can think of is to define a gateway as a collection of modules and have notes that observe the pecularites.

>  MUA Mailing Lists:
>
>  The actor for the Sender should be the intermediate submitter not the

For reference, I'm going to do away with the concept of "intermediate".  The reason is that it encourages folks to miss that a mailing list is a whole new ballgame for the message, with a new submitter.  Mailing lists are one example of a class of "user-level multi-hop process" functions that are part of group activity, including such fun stuff as organizational flow mechanisms like purchase approval".  I think the architectural reference needs to encompass this range of functions, rather than being tailored to mailing lists.  We are having quite a bit of confusion because of a constrained view of mailing lists as merely being extensions to the underlying handling service.

>  Whew. I think that's all.

good job.  thanks.  

now we get to debate...

d/
--
Dave Crocker
Brandenburg InternetWorking
+1.408.246.8253
dcrocker  a t ...
www.brandenburg.com

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, [email-masked]
Organization: Bruce Lilly
To: ietf-822 <[email-masked]>
Subject: Re: standards spring cleaning
Date: Tue, 7 Dec 2004 14:12:42 -0500
Cc: Nathaniel Borenstein <[email-masked]>, [email-masked]
Sender: [email-masked]

On Tue December 7 2004 12:29, Nathaniel Borenstein wrote:
> In both cases, I would be happier if we had 
> said something in the security considerations about the necessity to 
> reassemble message/partials into their "parent" message before applying 
> any security-related content scans.

Yes, but it's not solely a security matter.  Any scan of content
(for potential privacy violations or other leakage of information,
spam scanners, etc.) should take message fragmentation
(and external-body, embedded executable content, etc.) into
account.  Ultimately, it may be an intractable problem, since
"executable content" can take many forms due to the variety
of programming/scripting/macro languages available.

From: Nathaniel Borenstein <[email-masked]>
Subject: Re: standards spring cleaning
Date: Tue, 7 Dec 2004 12:29:26 -0500
To: ietf-822 <[email-masked]>, [email-masked]
Sender: [email-masked]

>> Don't forget Informational RFCs.  For example, RFC 1344 has
>> some very important advice for gateway implementors (and
>> that includes software which acts as a gateway, such as
>> message scanners).   Unfortunately some such implementors
>> have ignored 1344, resulting in problems:
>> http://www.kb.cert.org/vuls/id/836088

Actually, neither RFC 1344 nor RFC 2046, in my opinion, draw sufficient 
attention to this problem.  In both cases, I would be happier if we had 
said something in the security considerations about the necessity to 
reassemble message/partials into their "parent" message before applying 
any security-related content scans.

From: Bruce Lilly <[email-masked]>
Reply-To: ietf-822 <[email-masked]>, [email-masked]
Organization: Bruce Lilly
To: "ietf-822" <[email-masked]>, [email-masked]
Subject: Re: Fwd: standards spring cleaning
Date: Mon, 6 Dec 2004 12:43:43 -0500
Cc: Eliot Lear <[email-masked]>, "Paul Hoffman / IMC" <[email-masked]>, [email-masked]
Sender: [email-masked]

On Mon December 6 2004 10:32, Paul Hoffman / IMC wrote:

> >I am currently involved in an experimental process to see what it 
> >would take to hold to the notion that Proposed and Draft Standards 
> >shouldn't stay in that state forever.  There are a number of mail 
> >related standards that fall into this category, and I wonder if 
> >people can tell me whether the list of standards below (or any 
> >others on the broader list) should stay or go.  The original list 
> >was generated programmatically by looking for proposed standards 
> >below RFC 2000 that are not obsoleted (we'll do draft later).

Don't forget Informational RFCs.  For example, RFC 1344 has
some very important advice for gateway implementors (and
that includes software which acts as a gateway, such as
message scanners).   Unfortunately some such implementors
have ignored 1344, resulting in problems:
http://www.kb.cert.org/vuls/id/836088

Date: Mon, 6 Dec 2004 07:32:02 -0800
To: [email-masked], [email-masked]
From: Paul Hoffman / IMC <[email-masked]>
Subject: Fwd: standards spring cleaning
Sender: [email-masked]

[[ For those of you not on the original mailing lists this was sent to. ]]

>X-BrightmailFiltered: true
>X-Brightmail-Tracker: AAAAAA==
>Date: Mon, 06 Dec 2004 11:49:54 +0100
>From: Eliot Lear <[email-masked]>
>X-Accept-Language: en-us, en
>To: [email-masked], [email-masked], [email-masked]
>Subject: standards spring cleaning
>X-Enigmail-Version: 0.86.1.0
>X-Enigmail-Supports: pgp-inline, pgp-mime
>X-Spam-Score: 0.0 (/)
>X-Scan-Signature: 21c69d3cfc2dd19218717dbe1d974352
>Cc:
>X-BeenThere: [email-masked]
>X-Mailman-Version: 2.1.5
>List-Id: general discussion of application-layer protocols
>	<discuss.apps.ietf.org>
>List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/discuss>,
>	<mailto:[email-masked]?subject=unsubscribe>
>List-Post: <mailto:[email-masked]>
>List-Help: <mailto:[email-masked]?subject=help>
>List-Subscribe: <https://www1.ietf.org/mailman/listinfo/discuss>,
>	<mailto:[email-masked]?subject=subscribe>
>Sender: [email-masked]
>
>Good day,
>
>I am currently involved in an experimental process to see what it 
>would take to hold to the notion that Proposed and Draft Standards 
>shouldn't stay in that state forever.  There are a number of mail 
>related standards that fall into this category, and I wonder if 
>people can tell me whether the list of standards below (or any 
>others on the broader list) should stay or go.  The original list 
>was generated programmatically by looking for proposed standards 
>below RFC 2000 that are not obsoleted (we'll do draft later).
>
>Here they are:
>
>RFC1421       Privacy Enhancement for Internet Electronic Mail: Part 
>I: Message  Encryption and Authentication Procedures
>RFC1422       Privacy Enhancement for Internet Electronic Mail: Part 
>II: Certificate-Based Key Management
>RFC1423       Privacy Enhancement for Internet Electronic Mail: Part 
>III: Algorithms, Modes, and Identifiers
>RFC1424       Privacy Enhancement for Internet Electronic Mail: Part 
>IV: Key Certification and Related Services
>RFC1494       Equivalences between 1988 X.400 and RFC-822 Message Bodies
>RFC1496       Rules for downgrading messages from X.400/88 to 
>X.400/84 when MIME content-types are present in the messages
>RFC1502       X.400 Use of Extended Character Sets
>RFC1648       Postmaster Convention for X.400 Operations
>RFC1740       MIME Encapsulation of Macintosh Files - MacMIME
>RFC1767       MIME Encapsulation of EDI Objects
>RFC1848       MIME Object Security Services
>RFC1985       SMTP Service Extension for Remote Message Queue Starting
>               (ETRN)
>
>If you believe any of the above standards should not be reclassified 
>as historic, could you please send a note to 
>[email-masked] with the RFC number and a brief reason 
>as to why not and whether the document should be advanced.  A good 
>reason is that the standard is still generally useful.  A bad reason 
>is that a piece of old code implements it somewhere on the 'net. 
>That having been said we have a pretty low threshold for removing 
>something from the above list.
>
>Thanks for your time,
>
>Eliot

--Paul Hoffman, Director
--Internet Mail Consortium