Subject: Re: [Asrg] is SMTP still really store-and-forward ?

From: Dave CROCKER <[email-masked]>
Date: Mon, 01 Dec 2008 07:45:36 -0800

David Nicol wrote:
> On Fri, Nov 28, 2008 at 9:06 AM, John Levine <[email-masked]> wrote:
>> Also, SMTP is store-and-forward and [...] multiple hops.
> 
> It was designed that way, for UUCP bang-paths, but in practice for a

Email technology developed over a series of independent paths. (Feel free to 
take that as some sort of pun.)  The first inter-machine email was in 1971 by 
Ray Tomlinson of BBN, for the Arpanet.  Others for dial-up, such as Fidonet and 
uunet came later, as did an IBM-oriented service.  delivermail/sendmail was an 
early example of needing MTA functionality in order to support relaying. By 1980 
there were at least 4 independently developed multi-hop email services in 
operation.  Perhaps more.  The reference to "independently" means everything 
from concept to technology to operation.  Interconnections came quickly, but 
most had completely unrelated origins.

I have to make the pro forma reference to the Internet Mail Architecture 
specification:

    <http://bbiw.net/specifications/draft-crocker-email-arch-11.html>

(which is really and truly about to start through the IETF formalizations of 
standards approval, after 4 years of community discussion and modification. honest!)

Less pro-forma is to encourage looking at:

   <http://bbiw.net/specifications/draft-crocker-email-arch-11.html#relay>

which makes a distinction that is needed for this thread, namely that 
store-and-forward occurs at different levels.  And especially that email 
MTA-based "relaying" is not the same as re-posting by a Mediator such as a 
mailing list.  The examples cited on this thread have been for Mediator-based 
re-posting, not for SMTP-level relaying.

Since David raised the question about SMTP, the focus needs to be on the 
scenario from posting to delivery.  Are there multiple hops from the 
originator's posting machine to the recipient's delivery machine?

Arpanet/Internet mail has always tended to be dominated by the sort of "direct" 
scenario David cites, but "dominant" is not the same as "exclusive".  For 
example, larger organizations have department MSA/MDA servers that interact with 
border MTA servers. That's an internal hop.  It well might be repeated at the 
receiver's organization, making a common sequence of 3 SMTP relaying steps.

Some examples from mail sent on this thread...

John Day's posting from his machine to the mailing list:
> Received: from mail.ietf.org ([64.170.98.32])
> 	by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
> 	with ESMTP id IxjH0KqzpsGZ for <[email-masked]>;
> 	Sun, 30 Nov 2008 19:42:33 -0800 (PST)
> Received: from QMTA05.westchester.pa.mail.comcast.net
> 	(qmta05.westchester.pa.mail.comcast.net [76.96.62.48])
> 	by core3.amsl.com (Postfix) with ESMTP id EEA883A68A9
> 	for <[email-masked]>; Sun, 30 Nov 2008 19:42:32 -0800 (PST)
> Received: from OMTA08.westchester.pa.mail.comcast.net ([76.96.62.12])
> 	by QMTA05.westchester.pa.mail.comcast.net with comcast
> 	id le3Y1a03v0Fqzac55fiVCK; Mon, 01 Dec 2008 03:42:29 +0000
> Received: from [10.0.1.45] ([71.192.250.235])
> 	by OMTA08.westchester.pa.mail.comcast.net with comcast
> 	id lfiT1a01255V49j3UfiUfk; Mon, 01 Dec 2008 03:42:28 +0000

Michael Dillon's note:
> Received: from mail.ietf.org ([64.170.98.32])
> 	by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
> 	with ESMTP id mEVcGoR2p+P4 for <[email-masked]>;
> 	Mon,  1 Dec 2008 06:44:03 -0800 (PST)
> Received: from smtp4.smtp.bt.com (smtp4.smtp.bt.com [217.32.164.151])
> 	by core3.amsl.com (Postfix) with ESMTP id C09933A67F6
> 	for <[email-masked]>; Mon,  1 Dec 2008 06:44:02 -0800 (PST)
> Received: from E03MVZ2-UKDY.domain1.systemhost.net ([193.113.30.65]) by
> 	smtp4.smtp.bt.com with Microsoft SMTPSVC(6.0.3790.1830); 
> 	Mon, 1 Dec 2008 14:43:58 +0000

Stephane's note:
> Received: from mail.ietf.org ([64.170.98.32])
> 	by localhost (core3.amsl.com [127.0.0.1]) (amavisd-new, port 10024)
> 	with ESMTP id T6Dcuf3AWdw8 for <[email-masked]>;
> 	Sun, 30 Nov 2008 07:53:25 -0800 (PST)
> Received: from mail.bortzmeyer.org (aetius.bortzmeyer.org [217.70.190.232])
> 	by core3.amsl.com (Postfix) with ESMTP id 222A93A67B0
> 	for <[email-masked]>; Sun, 30 Nov 2008 07:53:24 -0800 (PST)
> Received: by mail.bortzmeyer.org (Postfix, from userid 10)
> 	id B52EB94AF6; Sun, 30 Nov 2008 16:53:18 +0100 (CET)
> Received: by mail.sources.org (Postfix, from userid 1000)
> 	id 928F2190701; Sun, 30 Nov 2008 16:49:19 +0100 (CET)

Each of these had MTA-level relaying from originator to the mailing list Mediator.

In more constrained networking environments, such as developing nations and 
networks supporting mobility -- especially for emergency services -- multiple 
hops at the email level is fairly common. Also for value-added uses, such as EDI 
and facsimile over email.

> very long time now, especially since running open relays began being
> poor form, SMTP has been sender-realm systems communicating directly
> to recipient-realm systems.
> 
> I believe that the requirement "support store-and-forward
> transparently" may be safely dropped from the needs of the FUSSP; 

      1. It's always worthwhile to review architectural assumptions.  If recent 
data confirms that an architecture is too complicated, it can be helpful to 
consider streamlining it.

      2. When doing that review, the dangers are in poor sampling or inadequate 
prioritization.  In other words, make sure your data cover the full range of the 
real world and make sure that you give proper weight to the inconvenient cases. 
For a global, diverse service is too easy to see only the most common cases, 
close at hand.

      3. Simplification reduces the potential for innovative uses.  Be careful 
not to lock out useful enhancements.

For all of its global scale, email technology is still pretty limited.  For 
example, notice the complete absence of any meaningful quality of service 
mechanism at the SMTP level.  Most of the real innovation in email is in how 
people lay human-operated logical layers on top of it, such as for for group 
collaboration and transaction processing. These, an others, have very rich 
functional requirements, in the non-email world, and their lack of support in 
email, IMO, constrains how we use email.  I tend to compare the email relaying 
service with the lower IP packet forwarding service and observe that QOS remains 
a hot concern down there.  It should be up for email, too.

Simply put, David, the global email transfer service is richer than you -- and 
many, many others -- appear to believe.

The fact that the most common scenario is a direct transfer between two huge 
servers is misleading.  Much of the recent discussions about anti-abuse have 
been distorted by the failure to appreciate the real and extensive diversity of 
email transfer scenarios we still rely on.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: Tony Hansen <[email-masked]>
Date: Mon, 01 Dec 2008 12:05:44 -0500

I was talking with someone in the past year or so who had been traveling
in southeast Asia. They experienced email on a totally different level:
it was dependent on a router being brought in to the area by a shared
traveling router. When the router was nearby, email flowed smoothly. But
until the router came back around on its weekly journey (carried on an
elephant, no less), the email had to be stored up before its final
delivery. Email definitely required delay-tolerant store and forward
processing.

	Tony Hansen
	[email-masked]

Dave CROCKER wrote:
> ...
> The fact that the most common scenario is a direct transfer between two
> huge servers is misleading.  Much of the recent discussions about
> anti-abuse have been distorted by the failure to appreciate the real and
> extensive diversity of email transfer scenarios we still rely on.

---

From: Dave CROCKER <[email-masked]>
Date: Mon, 01 Dec 2008 09:30:18 -0800

> delivery. Email definitely required delay-tolerant store and forward

Hadn't heard of that DTN example although some researchfield-monitoring 
scenarios are close.

The SMTP TRN (now ETRN) came out of our experience with CSNet's dial-up site 
access (phonenet) relaying to the Arpanet/Internet.  At one of the late-stage 
meetings that was designing SMTP, I mentioned this requirement for being 
connected occasionally and needing to pick up queued mail, as well as sending 
it.  When the remote site controlled the connection -- e.g., to save phone costs 
-- the other side would not know that it could initiate an SMTP push to the 
remote site.  Hence [E]TRN.

One of the fun example of Vint Cerf's talent was his immediately grasping the 
need to be able to explain the need for this, to funding agencies.  He was 
working at Arpa at the time and almost instantly said... submarines!

Actually, the most interesting thing is to hear that there are still some real 
work tasks for elephants.  In Thailand I'd heard that since they weren't needed 
for logging, entertaining tourists was the only thing keeping them alive...

d/

ps. a skeptic would note that your example doesn't explicitly state that the 
MTA-level is multi-hop, although I suppose there are MTAs on both sides of the 
occasional connection?
-- 

   Dave Crocker

---

From: "David Nicol" <[email-masked]>
Date: Mon, 1 Dec 2008 12:22:58 -0600

> originator's posting machine to the recipient's delivery machine?

I was not raising a question, I was making a proposal.  Alessandro
Vesely seems to be the only voice participating in this topic who
understood what I meant.

> appreciate the real and extensive diversity
> of email transfer scenarios we still rely on.

What I meant by "realm" is very close to "administrative domain."

Sending realm sends to receiving realm.  a backwards-compatible (no
zero day) FUSSP must live on the receiving realm.

My proposal, that I guess I did not make very well, is:

For purposes of discussing the nature of a FUSSP, operations within a
"realm" (which is not at this point a well-defined term) should be
treated as a black box that has full information sharing within it.
_______________________________________________
Asrg mailing list
[email-masked]
https://www.irtf.org/mailman/listinfo/asrg

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 03 Dec 2008 15:38:06 +0530

> to recipient-realm systems.

But, that's not the whole picture.  Your message went from gmail to IRTF to
me, which stored it until my MUA could grab it and store it for a few days
until I got around to deleting the postage thread.

Maybe we should call that forward-and-store instead, but the same core issue
remains: a true FUSSP must include the ability for the MUA to determine
stuff about the message without assuming permanent broadband network
connectivity at time of determination.

See also: draft-kucherawy-sender-auth-header, active & in a repository near
you.