Subject: [Asrg] SMTP pull anyone?

From: Ravi shankar <[email-masked]>
Date: Sun, 16 Aug 2009 16:50:39 +0530

Hi,

Me and my buddy had a interesting discussion, which i thought could put
across the geeks here.

It goes something like this:

SMTP is currently a push protocol and is initiated by the the sender, no
controlling that fact.

But it is possible to overcome the relay problems, IP spoofing and domain
impersonation etc,

by making the servers pull the mails.

1. Sending server contacts the destination and proovides the Message ID and
sender details(and other details) and disconnects the session.

2. The receiving server queues it up and looks up the messages one by one
using DNS to determine their legitimacy.

3. If the IP that contacted is legitimate(can be verified by say SPF?), it
contacts the sender and provides the message ID with other details.

4. The sending server then hands over the message.

5. To overcome DDoS attacks, the receiving server can be made to request the
next 10 or so Message IDs that it will assign to messages,

so that if a attacker tries to give those details, it will know from the
next list of message IDs that it's fake connection.

6. May be by this collection of data, the IP addresses can be reported to a
RBL and blacklisted.

Please point the holes in this model, so that we might get a entirely new
insight.

Note: I have gone trough IM2000 and other similar discussions in the
archive. Just thought this version of C/R is worth getting discussed.

Regards,
Ravi

---

From: Bill Cole <[email-masked]>
Date: Sun, 16 Aug 2009 14:14:00 -0400

Ravi shankar wrote, On 8/16/09 7:20 AM:
> But it is possible to overcome the relay problems, IP spoofing and
> domain impersonation etc,
> 1. Sending server contacts the destination and proovides the Message ID
> and sender details(and other details) and disconnects the session.
> 2. The receiving server queues it up and looks up the messages one by
> one using DNS to determine their legitimacy.

DNS itself provides no mechanism for determining the legitimacy of email.

DNS is used *as a medium* for various applications that are used to identify 
mail as legitimate or illegitimate by various standards of legitimacy, and a 
major reason for its use in those applications is to make it feasible for 
mail systems to do the validation synchronously during the SMTP session. By 
using a lightweight, distributed, cached database, mail systems are spared 
from deferring a message, queuing its validation, remembering the results, 
and waiting for the sender to offer it in an identical way again. You are 
suggesting that receivers should take on all the heavyweight management but 
retain using DNS for something unspecified. It makes no sense.

> 3. If the IP that contacted is legitimate(can be verified by say SPF?),
> it contacts the sender and provides the message ID with other details.

The *most* that SPF can provide towards showing "legitimacy" is to confirm 
that the envelope sender address of a message is not forged. It is very rare 
for large senders of any sort to deploy records that can do that strongly. 
There is nothing about SPF that directly attacks spamming. It could in 
theory be used to attack sender forgery, but the collateral damage has 
proven to be too great for either sending or receiving systems to actually 
apply it strongly to that end. Meanwhile, a lot of spammers are sending a 
lot of spam with senders that are validated to the degree that SPF can 
validate anything.

> 5. To overcome DDoS attacks, the receiving server can be made to request
> the next 10 or so Message IDs that it will assign to messages,

That sentence makes no sense. What did you mean to say?

> 6. May be by this collection of data, the IP addresses can be reported
> to a RBL and blacklisted.
> Please point the holes in this model, so that we might get a entirely
> new insight.

Nothing you have described would add to spam control as it is currently 
being done, as far as I can see. The 'model' is too vague to critique inn 
detail because you aren't really providing any meaningful details.

In order to bring anything truly new and useful to controlling email spam, a 
new idea has to either attack spam in a way that existing tactics don't, do 
a demonstrably better job than existing tactics, or overcome the negative 
aspects of existing tactics. You have identified none of those in your new 
idea.

> archive. Just thought this version of C/R is worth getting discussed.

As described, there's not much to discuss. It looks like it might be a 
little like IM2000 (which has been a complete failure) if fleshed out, but 
you haven't explained how this would differ from it, how it would integrate 
with existing SMTP, or how it would offer anything significantly better than 
existing mechanisms.

Right now, receivers can choose to validate the sender against SPF records 
after receiving the MAIL command and reject non-validating messages at that 
point, or they can wait until RCPT if they want their rejections to be more 
broadly respected, or they can wait until the second phase of DATA if they 
want to weight SPF results along with other filters that rely on message 
data. Many sites already explicitly reject the vast majority of offered 
messages in response to RCPT or earlier without requiring legitimate senders 
to sit on a pile of pending offered messages that may or may not be asked 
for later. You have not identified any benefits for receiving systems or 
legitimate senders, so it is hard to agree that it is "worth getting 
discussed" or indeed worth anything at all. Where is the added value?

---

From: mathew <[email-masked]>
Date: Sun, 16 Aug 2009 13:38:12 -0500

On Sun, Aug 16, 2009 at 06:20, Ravi shankar<[email-masked]> wrote:
> Me and my buddy had a interesting discussion, which i thought could
> put across the geeks here.
[... pull e-mail description...]

I wrote up something very similar, and as noted, there was IM2000.

At this point, the consensus on this list as far as I can determine is
"Nothing will work, and if you think otherwise go implement it and
prove us wrong". So I'd suggest digging out an ANSI C compiler as your
only productive next step.

mathew
-- 
<URL:http://www.pobox.com/~meta/>

---

From: Dave CROCKER <[email-masked]>
Date: Sun, 16 Aug 2009 11:45:50 -0700

> by making the servers pull the mails.

<http://v3.espacenet.com/textdoc?DB=EPODOC&IDX=US6192407>

-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: John Levine <[email-masked]>
Date: 16 Aug 2009 22:29:44 -0000

>> SMTP is currently a push protocol and is initiated by the the
>> sender, no controlling that fact.

> US6192407

Ah, right, the famous Tumbleweed patent.  Here's a better link
with a readable version.

http://www.google.com/patents?vid=USPAT6192407

This patent covers the idea of putting a message on a web server and
sending the recipient a URL to retrieve the message.  Tumbleweed
enforces it very aggressively.  See the list of suits in their
Wikipedia page.

Were it not for the patent, I expect we'd have a lot of institutions
setting up per-user password protected RSS or Atom feeds with
statements and other messages, and sending you mail telling you to
check your feed when there's something new.

R's,
John

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 17 Aug 2009 11:05:44 +0100

--On 16 August 2009 14:14:00 -0400 Bill Cole <[email-masked]> 
wrote:

> The *most* that SPF can provide towards showing "legitimacy" is to
> confirm that the envelope sender address of a message is not forged. It
> is very rare for large senders of any sort to deploy records that can do
> that strongly. There is nothing about SPF that directly attacks spamming.
> It could in theory be used to attack sender forgery, but the collateral
> damage has proven to be too great for either sending or receiving systems
> to actually apply it strongly to that end. Meanwhile, a lot of spammers
> are sending a lot of spam with senders that are validated to the degree
> that SPF can validate anything.

SPF deployment has grown rapidly from 5% of 2,000,000 sampled domains to 
17% over the past three years, apparently including most USA banks. About 
half of spf publishing domains, including some large senders like facebook, 
use "-all" records. Apart from anything else "-all" on its own is a good 
way of saying "this isn't an email domain", and it's probably a good idea 
to publish it for every A record that doesn't point to a mail server.

Furthermore, some large recipients like gmail use these records to help 
assign reputation to senders. Forwarded email is likely already suffering 
from deliverability problems when the sender address is not rewritten (at 
least in cases like forwarding facebook mail to gmail, for example), and 
these problems will continue to get worse, not better as SPF deployment 
increases, and as records are increasingly respected.

People who want to offer a reliable forwarding service to their users 
already need to be thinking about rewriting sender addresses.

It might take a few years, but I'm convinced that we'll look back on SPF 
deployment in the same was that we look back on the campaign against open 
relays some years ago.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: Michael Thomas <[email-masked]>
Date: Mon, 17 Aug 2009 09:34:27 -0700

I don't see how push or pull fundamentally changes the spam
equation in any case. The problem wrt spam is the any-any nature
of who you receive communication from, not who initiates the
connection.

I can create a walled garden trivially with smtp just be rejecting
connections from people not on my whitelist. Likewise, anything that
is promiscuous with who it receives from better be prepared for
mail-transmitted diseases (MTD's). That's just the nature of opening
up. So SMTP itself isn't the problem, or more specifically fretting
about SMTP is nipping around the edges of 20 with the 80-20 rule.

Mike

On 08/16/2009 03:29 PM, John Levine wrote:
> _______________________________________________
> Asrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/asrg

---

From: Douglas Otis <[email-masked]>
Date: Mon, 17 Aug 2009 10:14:44 -0700

> of who you receive communication from, not who initiates the

The pull concept is fairly simple.  When creating a white-list, it 
becomes important to identify the source of the message.  Since email 
uses store and forwarding, originating sources of messages can be 
difficult to determine, based upon the message alone.  Of course DKIM 
helps with that, but it would be more ideal to shift a greater portion 
of the burden toward the sender for email to continue to scale.

By exchanging only references to messages that are held on-line, the 
message itself does not need to be initially exchanged.  When contacted 
by a specific source that has been white-listed, your MUA could fetch 
the desired message from the on-line server at the same time references 
have been retrieved.

When a reference has been falsified, no message can be fetched.  This 
would not demand cryptographic efforts by the sender, or expect receipt 
of an exponentially increasing volume of junk, or tens or hundreds of 
DNS transactions per domain to resolve server authorizations.

One wonders whether IMAP might be tweaked to provide an online function 
rather than using traditional URIs.  The problem should not be viewed as 
the 80-20 rule where 80% of the problems are caused by 20% of the 
sources.  This should be viewed as the 99-99 rule, in today's email 
environment.

-Doug

---

From: John Levine <[email-masked]>
Date: 18 Aug 2009 03:47:36 -0000

>I don't see how push or pull fundamentally changes the spam equation
>in any case. The problem wrt spam is the any-any nature of who you
>receive communication from, not who initiates the connection.

I agree.  The main advantage I see to pull is bandwidth management,
since recipients can decide when and whether to retrieve mail for
users, rather than having to accept it all just in case someone wants
to read it.

The disadvantage is that it requires that senders and receivers all be
connected to the same network all the time, which is a lot closer to
true now than it was in the 1980s, but still a long way from
universally true.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 18 Aug 2009 17:40:20 +0400

> about SMTP is nipping around the edges of 20 with the 80-20 rule.

I don't think "opening up" is bound to have a unique nature. A 
protocol might provide for ways to open up _and_ allow 
traceability, responsibility, and accountability. Such features 
are implicit in walled gardens. Explicating them requires a more 
thorough analysis of the relevant conventions and rules, at an 
abstraction level such that its outcome can be globally 
accepted. Much of the required "political" work is already 
available as privacy recommendations --later than SMTP. It is 
possible to tweak SMTP so as to comply with that. While that 
would not "eliminate spam", it would permit to manage it 
cleanly, properly, and without sacrificing reliability.

---

From: Dave CROCKER <[email-masked]>
Date: Tue, 18 Aug 2009 07:24:25 -0700

> I agree.  The main advantage I see to pull is bandwidth management,
...
> connected to the same network all the time, 

The disadvantages are more substantial.  This is a very high overhead model, in 
terms of hassling the recipient and in terms of transactions costs.

d/

-- 

   Dave Crocker

---

From: Bart Schaefer <[email-masked]>
Date: Tue, 18 Aug 2009 01:01:36 -0700

On Aug 18,  3:47am, John Levine wrote:
>
> The disadvantage is that it requires that senders and receivers all
> be connected to the same network all the time, which is a lot closer
> to true now than it was in the 1980s, but still a long way from
> universally true.

I'm not a proponent of the pull model, but it seems to me this could
be overcome if each receiver in a store-and-forward sequence were
able to inform its immediate upstream when it is not able to perform
the pull.

E.g., transmit the "pull handle" as far as possible, then have the
last relay that is capable of doing so, issue the pull request.

This could probably even be done with one new SMTP verb plus ETRN.
E.g. DATA FROM:<message-id> where the domain part of the message-id
must resolve to the host that will cough up the message.  If the
receiver responds 50z or 455, the sender must do the pull.  If the
receiver responds 354, sender sends dot, and eventually some receiver
connects to <message-id-domain> and issues ETRN #<message-id-local>.

All sorts of security implications to be worked out, Tumbleweed
problems aside.