Subject: Re: [ietf-smtp] Idea: Two-Way Mail

From: Ned Freed <[email-masked]>
Date: Fri, 27 Feb 2015 17:03:02 -0800 (PST)

> On Wed 25/Feb/2015 16:35:33 +0100 Kai Engert wrote:
> >
> > I'm not sure what's the best place to send this idea to enhance SMTP for
> > avoiding spam. The ASRG list is no longer active, so I'm trying it on
> > this list. Please suggest better places if you think it's inappropriate.
> > I think it might be appropriate, if there's willinglist to adopt. I
> > understand that many people/groups would have to agree to such an
> > enhancement of today's SMTP, so let's start by discussing if this
> > approach could work.

> The reason why it is inappropriate is that the IETF seems to be committed to
> standardizing practices which are already in widespread use, rather than
> discussing new ideas.  For example, it recently elevated to Internet Standard
> the ASCII format for network interchange:
> http://mailarchive.ietf.org/arch/msg/ietf-announce/rlIsY8yvvKhbkbkuk2mcQ-ylSNw

Nonsense. John Klensin already explained why this action in regards to ASCII is
apropos of exactly nothing.

Moreover, the counterexamples to this thesis are thick on the ground: MIME was
in no way deployed when it was standardized, nor was/is EAI. HTTP/2 is the
obvious recent non-email counterexample. The list goes on and on and on.

The problem with most "new" anti-spam techniques is actually the exact
opposite: They aren't new, they often have been deployed as part of one system
or another, and when that was done they didn't work very well. (John Levine has
already responded to this particular proposal; I have nothing to add to that
response.)

Another problem is the IETF standardizes Internet protocols, not techniques for
doing things. A lot of anti-spam techniques have no protocol component, and
thus are not eligible for standardization. Just as one example, the technique
of whitelisting originators that appear in a user's address book is a local
matter that has no obvious protocol component.

Of course this doesn't mean informational RFCs couldn't be written. But we're
talking about standardization here, not RFC publication.

All that said, the IETF does have a substantial bias that impedes the adoption
of many of the anti-spam techniques I see mentioned on various lists: The IETF
is a lot happier when someone walks up with a reasonably complete specification
for whatever it is they want to do and then stands behind that specification.

Coming in with the equivalent of jottings on a napkin, or suggesting
substantive changes to someone else's work that substantively change the
character of that work, are not going to be welcomed unless the work has clear
and immediate utility.

And why should they be? It takes a lot of effort to get something onto the
standards track. (MIME took many hundreds of hours of Nathaniel's and my time.
I expect EAI took comparable amounts.) Expecting someone else to do all that
work simply isn't reasonable.

If you feel strongly that something needs to be a standard, then you need to
enact the labor necessary to get that to happen.

				Ned

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 03 Mar 2015 09:46:25 +0100

Hi Ned,

> apropos of exactly nothing.

C'mon, at least concede it was an awesome way to celebrate the International
Troll Day or whatever it was...

> already responded to this particular proposal; I have nothing to add to that

What drives me nuts is the fatalistic stance of assuming spam is an unavoidable
factor of our lives.  That is partly true, of course, but the requirement to
keep it within some boundaries seems to keep being overlooked in email
standardization.  Spam happens or happened to afflict web and DNS too, but in
those realms it is possible to devise solutions which work well enough.

For a different unavoidable factor, mass surveillance is the target of some
commendable IETF work.  Why is email spam less important?

For an anti-spam technique, authentication seems to be useful for nailing
spammers to their responsibilities.  It seems to be effective for avoiding
disruptive web contributions and "default" DNS replies.  Yet, for email, it
doesn't work very well.  I note that the likely author of 5321bis doesn't feel
a need to authenticate outgoing mail by deploying methods like DKIM, SPF, or
DNSWL.  I also wonder why the filter that has been keeping his mailbox clear
for decades is not globally deployed as an integral part of the protocol.

When each and every proposal has technical impediments and insurmountable
defects, one comes to think that the problem is ill-conditioned.  Like for
world hunger and global warming, there could be ways to overcome some of the
troubles, but pursuing them conflicts with unutterable needs which interfere
with any possible solution because of its very nature of attempting to remove
the problem, conspiracy theorists would say.  Customary attitudes range from
denying that the problem exists to contemplating how uninteresting it is to the
general public, as Krebs narrated in his last year's book.

OT and useless as this post may be, I'm not saying anyone should do anything
about spam.  I just doubt that your (not fully quoted) explanations capture the
essence of why nothing is being done.

Ale

---

From: Martijn Grooten <[email-masked]>
Date: Tue, 3 Mar 2015 10:06:33 +0000

On Tue, Mar 03, 2015 at 09:46:25AM +0100, Alessandro Vesely wrote:
> standardization.

I actually think that what is being overlooked by just about every new
anti-spam technique I've seen proposed is that we're already doing
rather well. Only a small minority of spam makes it to people's inboxes.
False positives do occur, but they're not exactly making people stop
using email.

So any improvement that can be made - and I'm not suggesting they cannot
be made - will be minor. Most proposals don't acknowledge that, while
they play down (or, as in the proposal that started this discussion,
completely ignore) the compromises that need to be made for the proposal
to work. I would say that dropping the ability to send unsolicited
emails is a huge compromise.

False positives tend to be more common (and much harder to avoid) on
emails from senders that one hasn't previously communicated with. So
this proposal 'solves' this problem by saying these emails shouldn't
have been sent in the first place.

Moreover, there is a growing number of ways to send messages using apps
that require you to implicitly pre-approve the sender (social network
DMs, various messaging apps). I'm no expert on them, but they tend to be
built with security in mind and offer really good encryption. Some of
them apparently seamlessly work together.

Perhaps the IETF can play a role there by fixing some standards. And
perhaps in a few years' time, everyone will have ditched email in favour
of such apps/protocols.

In the meantime, I think it would be good to find ways to solve (i.e.
mitigate) the problem of email spam within the context of SMTP.

Martijn.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 3 Mar 2015 11:05:46 +0100

Alessandro Vesely:
> Why is email spam less important?

You assume that lack of importance is the (major) cause of inaction. But 
John Klensin already told you a very different reason why he isn't going on 
with 5321bis: Do anything with the core of the mail system (and spam is at 
the core now), and advokids drown you in arguments. It's not that mail is 
objectively unimportant or that John thinks it's unimportant, it's that 
whoever writes the RFC pays a high personal price.

John wrote much more. My eyes glazed over. But I think this is the MSB.

Arnt

---

From: John C Klensin <[email-masked]>
Date: Tue, 03 Mar 2015 09:34:54 -0500

--On Tuesday, March 03, 2015 11:05 +0100 Arnt Gulbrandsen
<[email-masked]> wrote:

> You assume that lack of importance is the (major) cause of
> inaction. But John Klensin already told you a very different
> reason why he isn't going on with 5321bis: Do anything with
> the core of the mail system (and spam is at the core now), and
> advokids drown you in arguments. It's not that mail is
> objectively unimportant or that John thinks it's unimportant,
> it's that whoever writes the RFC pays a high personal price.
> 

For that part of the issue, yes.  I would have stated it as "the
apparent value is, so far, far exceeded by that high price"
(not just personal, but to the community).   The reason I'm
keeping a running editor's update is precisely because I imagine
that ratio might change someday and want to be able to respond
quickly.

In the interim, a variation on one of Ned's comments apply:
5321bis will be strictly about corrections and clarifications,
incorporating substantive changes only if they are mature in
both practice and IETF procedures.  So, if someone wants a
changed way of doing something as part of it, write an I-D
specific to that protocol and see if it goes through the system
and deploys.

   john

---

From: John C Klensin <[email-masked]>
Date: Tue, 03 Mar 2015 10:03:13 -0500

--On Tuesday, March 03, 2015 10:06 +0000 Martijn Grooten
<[email-masked]> wrote:

>... 
> I actually think that what is being overlooked by just about
> every new anti-spam technique I've seen proposed is that we're
> already doing rather well. Only a small minority of spam makes
> it to people's inboxes. False positives do occur, but they're
> not exactly making people stop using email.
>...
> In the meantime, I think it would be good to find ways to
> solve (i.e. mitigate) the problem of email spam within the
> context of SMTP.

Martijn,

There is one other issue, of which, IMO, we keep losing sight.
Filter-based (or similar) spam mitigation is effective at
keeping recipients from being overwhelmed by spam volume and
reasonably effective at preventing spam-enabled malware, etc.,
from being delivered.  For places with restricted or expensive
available bandwidth to delivery (or filtering) servers, it is
ineffective at reducing the impact on transport bandwidth.
Indeed, at some times, if filters block 90% of the spam,
spammers simply increase the volume in the hope of getting
desired absolute amounts through, and typically through to the
most vunerable recipients.   That combination turns high-volume
spam into a DoS or imposition of costs attack, something we have
heard a lot about, especially from developing countries in
internal "governance" and "cybersecurity" meetings.

How significant that spam volume is relative to the average
movie download is another question, but the perception is that
it is important (and that few people with really restricted or
expensive links are downloading movies anyway).

Solutions to that class of problem lie in methods that keep the
spam off the network rather than keeping it from being delivered
once it is transported.  Those solutions, so far, seem to be
more social or legal than technical.

Ironically, the better we get at spam mitigation, the less
likely we are to see sociopolitical remedies (such as throwing
spammers in jail and keeping them there) because the key
decision-makers --who typically have better access to good
filtering and bandwidth than typical members of their
populations-- have sufficiently little spam delivered to them
that they don't see the problem as significant.

So mitigation is not entirely a good thing.

"Two-way" techniques, whether via this sort of proposal or via
arrangements like one very common in the financial industry
(sending a very short message that says "you have secure mail,
log into your web account to pick it up") at least reduce the
bandwidth used by unsolicited or undesired mail.  On the other
hand, a de facto policy that amounted to "spam is ok as long as
the messages are short" would not really constitute progress
either.

    john

---

From: Dave Crocker <[email-masked]>
Date: Tue, 03 Mar 2015 21:13:45 -0800

On 2/27/2015 5:03 PM, Ned Freed wrote:
> opposite: 

This fact was nicely institutionalized, long ago, by Cory Doctorow:

     https://craphound.com/spamsolutions.txt :

     Your post advocates a

     ( ) technical ( ) legislative ( ) market-based ( ) vigilante

     approach to fighting spam. Your idea will not work. Here is why it
     won't work.
     ...

d/
-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Martijn Grooten <[email-masked]>
Date: Wed, 4 Mar 2015 21:21:46 +0000

> ineffective at reducing the impact on transport bandwidth.

I didn't mean just filters, I meant to include legislation, botnet
takedowns, outbound filtering, DNS blacklists etc. all of which do
greatly mitigate spam as a DoS attack on the global email
infrastructure. (You can add hosted spam filters to that list as a
mitigation of spam eating your bandwidth.)

Martijn.

---

From: Дилян Палаузов <[email-masked]>
Date: Fri, 06 Mar 2015 12:25:04 +0100

Hello Kai,

in the case where a web-mailbox is hacked and used to send spam to all 
contacts in that mailbox, those contacts using two-way mail would get 
spam, as the mechanism, proposed by you, negates the effect of 
conventional anti-spam systems.

In this particular case, not-using two-way mail will lead to less 
received spam.

Mailboxes get hacked, as users use one and the same password to register 
everywhere in internet and often they leave their email address to all 
those registrations.  So the providers of such platforms (e.g. a forum) 
have email address and password, which probably match.

Greetings
   Dilian

On 25.02.2015 16:35, Kai Engert wrote:
> Hello,
> A (draft) writeup of this idea can be found as a PDF file at:
> https://kuix.de/two-way-mail/
>
> Very short summary below:
> =================================
> Email should adopt the both-way opt-in requirement used by many instant
> messaging systems, before messages are accepted for delivery.
>
> Require email address whitelists, controlled by the user, managed on the
> user's mailbox server.
>
> Find a way to signal contact attempts to the recipient, enabling a user
> to discover them and add entries to the whitelist.
>
> Change email delivery from today's store-and-forward approach and switch
> to store-notify-and-poll.
>
> Have the destination server retrieve email only if the sender address is
> in the recipient's whitelist.
>
> Have the destination server contact a mail server based on official
> records (e.g. DNS), instead of accepting mail from anywhere.
> =================================
>
> Please let me know in case this is actually an old idea and has been
> discussed/rejected already.
> I'd appreciate your feedback.
>
> Best Regards
> Kai
>
>
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp
>

---

From: Douglas Otis <[email-masked]>
Date: Fri, 6 Mar 2015 09:59:55 -0800

> On Mar 3, 2015, at 7:03 AM, John C Klensin <[email-masked]> wrote:
> 

Dear John,

Agreeing with Martijn.  The Internet will reach a tipping point for IPv6.  Diehards insist this will never happen for email, however use of infrastructure identifiers rapidly becomes overwhelming when over IPv6 or proxied into IPv4 through massively shared infrastructure.  What will having an rDNS PTR record really mean?  The malefactors are upping the game with BGP injections. For example, can there be a standard that support SMTP where both the client and server are confirmed such as DANE-DANE/SMTP?

Regards,
Douglas Otis

---

From: John C Klensin <[email-masked]>
Date: Fri, 06 Mar 2015 13:43:50 -0500

--On Friday, March 06, 2015 09:59 -0800 Douglas Otis
<[email-masked]> wrote:

> Agreeing with Martijn.  The Internet will reach a tipping
> point for IPv6.  Diehards insist this will never happen for
> email, however use of infrastructure identifiers rapidly
> becomes overwhelming when over IPv6 or proxied into IPv4
> through massively shared infrastructure.  What will having an
> rDNS PTR record really mean?  The malefactors are upping the
> game with BGP injections. For example, can there be a standard
> that support SMTP where both the client and server are
> confirmed such as DANE-DANE/SMTP?

Nothing above is really inconsistent with my comments -- I was
just pointing out another aspect of the spam-specific issue.
Partially for those reasons, I believe that it is unlikely that
a redesigned and even somewhat incompatible mail system will be
deployed and get significant use on the basis of spam mitigation
alone, but I certainly would not try to prevent anyone from
trying.

Write a proposal, in the usual I-D form and with all of the
relevant details, including a transition strategy, and see if
there is sufficient interest to move it forward.   Or implement
your ideas and try them out in the marketplace.  To paraphrase
some comments from long ago, the shoulders of the information
superhighway are littered with proprietary email solutions that
did not interoperate smoothly with SMTP, but perhaps the next
one will be more lucky.

best,
   john