Subject: [Asrg] is SMTP still really store-and-forward ?

From: "David Nicol" <[email-masked]>
Date: Fri, 28 Nov 2008 13:21:02 -0600

On Fri, Nov 28, 2008 at 9:06 AM, John Levine <[email-masked]> wrote:
> Also, SMTP is store-and-forward and [...] multiple hops.

It was designed that way, for UUCP bang-paths, but in practice for a
very long time now, especially since running open relays began being
poor form, SMTP has been sender-realm systems communicating directly
to recipient-realm systems.

I believe that the requirement "support store-and-forward
transparently" may be safely dropped from the needs of the FUSSP; I
believe that systems that forward without rewriting the envelope
return address are to be considered broken.  I therefore prefer SPF to
DKIM, while respecting that DKIM works and does handle forwarding
better.

-- 
signature closed for repair
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: John Levine <[email-masked]>
Date: 28 Nov 2008 20:06:27 -0000

>It was designed that way, for UUCP bang-paths

I used UUCP for a long time, and I can assure you that SMTP's design
had nothing to do with bang path hacks.  Please do your homework.
Dave Crocker, who wrote many of the relevant documents, might be able
to point to some useful references.

> but in practice for a very long time now, especially since running
>open relays began being poor form, SMTP has been sender-realm systems
>communicating directly to recipient-realm systems.

Hmmn.  This might be a good time to learn more about the way that
non-trivial mail systems work.  There is a lot of multi-hop SMTP,
and it has nothing to do with open relays.

R's,
John

---

From: "Chris Lewis" <[email-masked]>
Date: Fri, 28 Nov 2008 15:17:05 -0500

> DKIM, while respecting that DKIM works and does handle forwarding

Your list article traversed _four_ store-and-forward hops after leaving
the IETF mail server before getting to me - and those are just our
internal MTAs.  And it looks like it went thru several store-and-forward
hops getting from your account on gmail to the IETF, and a couple inside
the IETF.

There's one heck of a lot of Received lines in there...

Store-and-forward is going to be with us for quite a while yet, even
without Internet open-relays.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Fri, 28 Nov 2008 12:37:59 -0800

>to recipient-realm systems.

When I send out this message, it is submitted to a MSA where it is 
queued.  The message is then relayed to the MX host for irtf.org 
where it gets delivered to the recipient.  Section 3.9.2 of RFC 5321 
explains how the message gets redistributed to the subscribers of 
this mailing list.  There is some store and forward being done.  The 
SMTP server cannot take full responsibility for a message without doing that.

There have been several proposals about moving away from the 
store-and-forward model.  Some of these proposals assume that there 
is direct connectivity between the sending and receiving 
systems.  That's not the case.  A message can go through several hops 
before being delivered to the recipient.  It's not uncommon to see 
that nowadays when mail filtering has to be performed.  Although 
there isn't a lot of source routing being done, people still use 
routing tricks even though it's not transparent to the sender.

Regards,
-sm

---

From: "David Nicol" <[email-masked]>
Date: Fri, 28 Nov 2008 15:03:06 -0600

> hops getting from your account on gmail to the IETF, and a couple inside

I said "realm" not "host." From the list to all of our in-boxes, the
envelope return address was politely changed to
"[email-masked]" by Mailman version 2.1.9.

Figure 1 in rfc821 is the SMTP model, and it does not involve
intermediate hops.  Complex multi-host systems one one side or the
other collapse into "Sender SMTP" and "Receiver SMTP" in that diagram.

gmail realm <-> irtf realm <-> nortel realm

Two transmissions between realms.

I meant "realm" not "host" which is what I said.  If there's a better
term for what I mean by "realm" I am open to changing my terminology.

In arguing to drop store-and-forward as a requirement, the internal of
operations of a realm are considered a black box outside the FUSSP's
scope.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Michael Thomas <[email-masked]>
Date: Fri, 28 Nov 2008 13:05:07 -0800

SM wrote:
> store-and-forward model.  Some of these proposals assume that there is 
> direct connectivity between the sending and receiving systems.  That's 
> not the case.  A message can go through several hops before being 
> delivered to the recipient.  It's not uncommon to see that nowadays 
> when mail filtering has to be performed.  Although there isn't a lot 
> of source routing being done, people still use routing tricks even 
> though it's not transparent to the sender.
The question I have is what problem is trying to be solved even if one
were inclined to move away from store and forward. It's not very clear
that there is one.

       Mike

---

From: "David Nicol" <[email-masked]>
Date: Fri, 28 Nov 2008 15:17:49 -0600

> were inclined to move away from store and forward. It's not very clear

The problem is redefining large portions of e-mail operations outside
the scope of the FUSSP.

Just as internal ops are disincluded from RBL filtering, internal ops
should be left out of any other technique.

---

From: Steve Atkins <[email-masked]>
Date: Fri, 28 Nov 2008 14:10:53 -0800

> operations of a realm are considered a black box outside the FUSSP's

If there is store-and-forward at any point along the path, that'll break
most toy ideas that are broken by store-and-forward.

If your toy idea is not like those other toy ideas then you should  
really
talk about the details, rather than asserting that the store-and-forward
that's commonly in use is not really store-and-forward.

Or, at least, define terms better. As one example of that, are you  
asserting
that the recipient MX always knows everything there is to know about
whether an email will be successfully delivered or not, such that all
the store and forward hops beyond that can be safely ignored?

Cheers,
   Steve

---

From: "David Nicol" <[email-masked]>
Date: Fri, 28 Nov 2008 17:29:53 -0600

On Fri, Nov 28, 2008 at 4:10 PM, Steve Atkins <[email-masked]> wrote:

> If your toy idea is not like those other toy ideas then you should really
> talk about the details

I've sworn off describing and defending my toy idea on this mailing
list, at least until it is operating again.

> Or, at least, define terms better. As one example of that, are you asserting

There are some assertions that I think are true and which apparently
aren't obvious, such as

1:  operations internal to a realm are outside the scope of proposals
2:  messages should either be rejected at the gateway or filed for review
3:  no additional SMTP messages should be generated by operation of the system

1 is a principle that appears in the implementations section of the
RBL document as a suggestion that one list internal MTAs as
exceptions; 2 happens now, mostly, with "spam folders" and 3 is
violated by challenge/response schemes.

#2 answers your question.  I do not assert that "the recipient MX
always knows everything there is to know about whether an email will
be successfully delivered or not" if "successfully delivered" means
"reaches the recipient's eyeballs" but I do believe that a recipient
MX has enough information to make an accept/trylater/reject decision,
which is what SMTP requires it to do.

In this thread, which I started, I wish to argue for leaving
store-and-forward done internal to an e-mail realm, both inbound and
outbound, out of future discussion as an eminently handwavable
implementation detail.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Fri, 28 Nov 2008 16:12:23 -0800

>1:  operations internal to a realm are outside the scope of proposals

Could you define "realm"?  Is that similar to ADMD?  A message can be 
relayed through several administrative entities before final 
delivery.  These entities might only have weak relationships between 
then.  I don't think it's possible to segment the entire delivery 
path into two parts.

>3:  no additional SMTP messages should be generated by operation of the system

That rules out delivery status notifications.  Such notifications 
aren't necessarily bounces.

>which is what SMTP requires it to do.

Could you point me to the relevant section of RFC 5321 which 
describes that requirement?

Regards,
-sm

---

From: "David Nicol" <[email-masked]>
Date: Fri, 28 Nov 2008 18:29:58 -0600

On Fri, Nov 28, 2008 at 6:12 PM, SM <[email-masked]> wrote:

>> I do believe that a recipient
> Could you point me to the relevant section of RFC 5321 which describes that
> requirement?

that would be section 4.2.1.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: SM <[email-masked]>
Date: Fri, 28 Nov 2008 16:59:15 -0800

>that would be section 4.2.1.

That section of RFC 5321 is about "Reply Code Severities and 
Theory".  It doesn't apply to what you said.  The figure from Section 
2.1 of RFC 5321 is about the basic structure of the SMTP model.  As 
the name implies, it's basic and it should not be confused with 
common SMTP environments.

I suggest that you read Section 3.3 of RFC 5321 that discusses mail 
transactions.  That RFC does not describe the email environment.  It 
specifies how the SMTP protocol works and offers guidance to implementors.

Regards,
-sm

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 29 Nov 2008 12:58:44 +0100

SM wrote:
> relayed through several administrative entities before final delivery.  
> These entities might only have weak relationships between then.  I don't 
> think it's possible to segment the entire delivery path into two parts.

If you follow the path of a given message, you find exactly one point 
where an MTA looks up the global DNS, determines a target host in the 
recipient domain's MX, and delivers to one of them. The boundary 
between the two realms lays in the middle of that hop. (Further hops 
from a backup MX to a higher priority one are conventionally in the 
recipient's realm.)

That doesn't necessarily coincide with an ADMD.

In case the message is further forwarded, e.g. this message, there 
must be a host who holds the additional piece of data that I call a 
forwarding recipe(*). The next segment is part of the recipient realm, 
if the target address in the recipe is being used with the permission 
of its owner.

[*]  http://fixforwarding.org/wiki/forwarding_recipe

---

From: SM <[email-masked]>
Date: Sat, 29 Nov 2008 08:47:52 -0800

At 03:58 29-11-2008, Alessandro Vesely wrote:
>forwarding recipe(*). The next segment is part of the recipient 
>realm, if the target address in the recipe is being used with the 
>permission of its owner.

The "forward" in "store-and-forward" is not the same as 
forwarding.  The concept is not limited to the Internet mail 
architecture.  It preserves data integrity and does not require full 
connectivity.  Let's say you have a mobile device running a mail 
service.  As you are moving, the device may detect a communication 
point which could be used for a message transfer.  It's somewhat like 
cell phone roaming with the difference that it doesn't require prior 
agreement to communicate.

Your forwarding recipe prescribes that a DSN must follow the reverse 
of the path of the original message.  It prevents having asynchronous 
paths which are quite common in communication networks.  That 
forwarding fix reminds me of source routing.

Regards,
-sm

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 29 Nov 2008 18:39:48 +0100

SM wrote:
> Your forwarding recipe prescribes that a DSN must follow the
> reverse of the path of the original message.

I don't think so. It _may_ do that, or it may just drop any DSN 
requirement for privacy considerations, or use ORCPT. It should depend 
on users preferences.

> It prevents having asynchronous paths which are quite common in
> communication networks.

I wouldn't like that. Actually, if a forwarding agreement were set up 
between a secondary MX and its higher priority servers, for each 
recipient, that would imply building a copy of the users database. 
Something that backup MXes ordinarily don't do.

> That forwarding fix reminds me of source routing.

Yes, while rfc 1123 solved the sending issue, it broke the reverse 
path. However, except for the case of backup MXes (that I haven't yet 
worked out well), forwarding implies altering the recipient address.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg