Subject: [ietf-smtp] Dombox - A Zero Spam Mail System

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 07:36:38 +0530

Hello everyone,

I would like to present my work here. It’s a new but backward compatible
mail system. It addresses some of the issues we face in the current email
system. E.g. Email Spam, Phishing, Privacy, Data Breach, Bandwidth Wastage
etc.

I designed the system in a way to deal with spam mails without wasting
network bandwidth. So it’s much more efficient than the current mail system.

Here is my white paper <https://www.dombox.org/dombox.pdf> I published in
Feb 2019. I have also published a summary
<https://medium.com/@Viruthagiri/dombox-the-zero-spam-mail-system-2b08ff7432cd>
of my system in a medium blog post.

------

When someone is trying to present a solution for a challenging problem like
email spam, skepticism is quite normal especially when the person is
someone you are not acquainted with. So I’m not expecting any welcome party
here. However, please stop engaging in personal attacks. Few months back,
In one of the mailing list I went completely nuts because one man kept
targeting me and engaging in personal attacks.

------

Coming to my paper, It’s a 300 page paper. It contains 154 figures. Most of
the figures occupies the whole page. So 50% of the content is filled with
figures.

In my early years I couldn’t find any beginner friendly material. I had a
very hard time in understanding stuff like SPF, DKIM, DMARC etc. It was too
complicated for me at that time. So the content in my paper is prepared for
a beginner. Most parts are explained in a very simplified way. Since most
of the people in this mailing list can be termed as “experts”, you may find
my paper as “too detailed”. My apologies for that.

In hindsight, I have realised some parts in my paper are unnecessary and
lame. I’ll probably remove those parts and improve the content in the
future. You may have to excuse me for those silly parts for the time being.

------

Apple also seem like copied some parts of my work from my white paper and
released it as “Sign In With Apple
<https://developer.apple.com/sign-in-with-apple/>”. Apple is either
stealing my work by misinterpreting some parts in the prior art OR all my
efforts related to “Sign In With Apple” is already in the public domain. I
have explained what’s going on between me and Apple in two blog posts. First
blog post
<https://medium.com/@Viruthagiri/apple-seem-like-copied-my-work-as-sign-in-with-apple-a309b19a24c2>
explains the similarity between my work and “Sign In With Apple”. Second
blog part
<https://medium.com/@Viruthagiri/here-is-why-apple-copied-my-work-as-sign-in-with-apple-6d35f400f82d>
explains why they are going for it. If you are from a company that has an
auth button like “Sign In With Apple”, then you might want to take a look.
You probably can integrate the “proxy” feature just like Apple if my work
is really in the public domain.

Thanks and Good luck.

-- 
Best Regards,

Viruthagiri Thirumavalavan
Dombox, Inc.

---

From: "John Levine" <[email-masked]>
Date: 24 Sep 2019 22:30:51 -0400

In article <CAOEezJSK39=[email-masked]> you write:
>mail system.

It looks a lot like Zoemail.  See US patent 5,930,479, filed in 1996
and issued in 1999.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 08:13:43 +0530

Thanks John. I'll take a look at it.

On Wed, Sep 25, 2019 at 8:00 AM John Levine <[email-masked]> wrote:

> In article <CAOEezJSK39=
> [email-masked]> you write:

-- 
Best Regards,

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 08:22:04 +0530

Seem like I have already disclosed that patent in my IDS
<https://www.dropbox.com/s/bczz1ccw5xj9k4w/ids.pdf?dl=0>.

-- 
Best Regards,

---

From: "John R Levine" <[email-masked]>
Date: 24 Sep 2019 23:06:58 -0400

> <https://www.dropbox.com/s/bczz1ccw5xj9k4w/ids.pdf?dl=0>.

That's fine, but your scheme still seems a lot like Zoemail.

Zoemail was invented over 20 years ago and although it worked technically, 
it was a complete failure as a mail system that people actually used.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 10:16:13 +0530

I just had a look at Zoemail.

To quote this page
<https://www.smallbusinesscomputing.com/biztools/article.php/3347011/ZoEmail-Has-Zero-Tolerance-for-Spam.htm>

> Your ZoEmail address is ultimately different for each person who sends you
> mail.
>

> It consists of a conventional user ID and domain name, with the ZoEmail
> key, an alpha-numeric string, wedged in before the @ sign — eg.
> [email-masked], "rcu99llp" being the key.
>

> The system automatically assigns randomly generated keys to legitimate
> correspondents and stores them in a look-up table. If an incoming message
> does not include a valid key, the system discards it, assuming it's spam.

In my opinion, the reason they were failed is because they were trying to
use the disposable email address feature for human-to-human mails. An email
address should be easy to give. It should be a single identity for
human-to-human mails. e.g. [email-masked].  It should be easily fillable
in an offline form.

In my system we are isolating only the website related mails.
human-to-human mails gonna work as usual.

-- 
Best Regards,

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 25 Sep 2019 09:55:43 +0200

On Wed 25/Sep/2019 04:06:38 +0200 Viruthagiri Thirumavalavan wrote:
> Here is my white paper <https://www.dombox.org/dombox.pdf>I published in Feb 2019.

    Bad guy forge the mail like it’s coming from [email-masked].
    Challenge mails are being sent to [email-masked]

That's gonna be tagged as yet another witch hunt...

Best
Ale
--

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Wed, 25 Sep 2019 04:07:14 -0400

> human-to-human mails gonna work as usual.

And you tell that it's a "website related" mail, how, exactly?

You get a connection on port 25. How do you know it's really a real person and
not a "website related" mail that lies in the From: field?

It's *really* easy to design a mail system that stops spam when everybody follows
the rules.   Making one that works even when the bad guys intentionally break the
rules is a lot harder...  And making one that works *and* that Joe Sixpack will actually
use is damned near impossible.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 15:06:33 +0530

> not a "website related" mail that lies in the From: field?

The system offers different address structures.  We decide how to treat the
mail based on the RCPT TO address. If it is look like RCPT TO: <
[email-masked]>, then it's a human-to-human mail address. If it looks
like  RCPT TO: <[email-masked]>, then it's a website
related mail address.

By default, [email-masked] is a generic mail address. It can accept mails
from both human and websites.  A user have to enable a setting called
"Restricted Mode" to instruct the system that it's a human-to-human mail
address. We heavily rely on MX record instead of SPF record to detect mail
genuinity in human-to-human mails.

-- 
Best Regards,

---

From: Richard Clayton <[email-masked]>
Date: Wed, 25 Sep 2019 11:02:56 +0100

In message <[email-masked]>, Alessandro
Vesely <[email-masked]> writes

>That's gonna be tagged as yet another witch hunt...

There is also an underlying assumption that when the challenge email is
received then the question it poses will be answered honestly

It has long been my policy when receiving such emails to always answer
that "yes I sent this email" -- whether or not this is true

My hope is that either the person trying out challenge response will
realise that dumping their spam filtering costs onto strangers is unwise
or they will conclude, incorrectly, that I am a spammer and block all
further email "from" me.  Either way they don't bother me again and I am
happy, so the effort of responding to them has been worthwhile

- From time to time volumes of challenge response email have been high,
for example when Earthlink used this type of system -- and so I
automated the sending of the responses ... they did have CAPTCHAs to
discourage such automation, but since they only had 31 CAPTCHA images
this didn't slow down the automation all that much

https://www.lightbluetouchpaper.org/2006/02/12/earthlink-has-just-
31-challenge-response-captchas/

- -- 
richard                                                   Richard Clayton

Those who would give up essential Liberty, to purchase a little temporary 
Safety, deserve neither Liberty nor Safety. Benjamin Franklin 11 Nov 1755

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 15:59:44 +0530

> that "yes I sent this email" -- whether or not this is true

In my system, challenge/response methods applicable only for "verified
strangers".  When the MAIL FROM says that the mail is coming from
[email-masked], our system going to fetch the MX record and check whether
the mail is really coming from example.com. Since we are talking about
human-to-human mails here, we are expecting the mail from one of your MX
servers. We also check SPF record and A record. If the mail is not coming
from any of those IP addresses, we actually reject the mail.

So the first step of validation is checking whether the incoming mail is
from a verified stranger.

Only then you are going to get the challenge mail. The whole point of
challenge/response here is to slow down spammers who genuinely buy domains
and send spam.

So the C/R mechanism in my system is not to check whether the mail is from
you. It's to slow down spammers who are sending mails from their MX record
IP addresses.

On Wed, Sep 25, 2019 at 3:33 PM Richard Clayton <[email-masked]>
wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> >> Here is my white paper <https://www.dombox.org/dombox.pdf>I published
> in Feb
> >2019.
> >
> <https://www.lightbluetouchpaper.org/2006/02/12/earthlink-has-just-31-challenge-response-captchas/>
>
> - --
> richard                                                   Richard Clayton
>
> -----BEGIN PGP SIGNATURE-----
> Version: PGPsdk version 1.7.1
>
> iQA/AwUBXYs7UDu8z1Kouez7EQJpkwCcDSDn2BnJ8FZ8OxTO5NUQILvUbJQAnidr
> 4oFkmT0lcYi4hyNALOMZJErv
> =ILXD
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp
>

-- 
Best Regards,

---

From: Дилян Палаузов <[email-masked]>
Date: Wed, 25 Sep 2019 10:45:34 +0000

Hello,

I print on a business card my address [email-masked] and give it to humans.  The humans store the email addresses in a
database.  The database gets stollen and thus my address [email-masked] gets stollen.  Then a “website” sends emails to 
[email-masked] and the algorithm below says this is a human-to-human communication.

This scenario is not so rare.  Users use the same username and password for different service providers.   Once data
from any service provider is leaked, which happes regularly, that credentials are tried as login on well-known email
providers and then all contacts stored in the addressbook of that email provider get spam, from a mailbox they know (so
no challenge emails).  Even if the mailbox is not used to send spam, the hacker has the [email-masked] address and the
new mail system cannot say anymore if the sender is human or bot.

As I said, first send emails from domains, whose MX records point to your own servers, then propose improvements to the
global mail architecture.

> When the MAIL FROM says that the mail is coming from [email-masked], our system going to fetch the MX record and
check whether the mail is really coming from example.com.

You cannot know, if the sending site is entitled to send emails from example.com by looking the MX records.  MX records
are for setting the hosts to receive emails, not for fixing the hosts, that send emails for a domain.

Returning challenge emails is nothing new, and is not a solution.  It imposes delays, and bothers the sender, just like
captchas bother the humans.  It requires additional concentration from the sender, who is supposed to answer the
challenge.  In any case, as far as this can be set up per recipient and the recipient decides to bother the senders with
email challenges, it is something that can be clarified between senders and that recipient.

Another way to slow down spammers, is to return delayed OK on EHLO, RCPT, MAIL FROM: and DATA END-DOT commands, with the
later not delivering the email, if the sender closed the connection before seeing OK.  Then spammers will have less
resources to send spam for the same unit of time.

Regards
  Дилян

On Wed, 2019-09-25 at 15:06 +0530, Viruthagiri Thirumavalavan wrote:
> The system offers different address structures.  We decide how to treat the mail based on the RCPT TO address. If it is look like RCPT TO: <[email-masked]>, then it's a human-to-human mail address. If it looks like  RCPT TO: <[email-masked]>, then it's a website related mail address. 
> 
> By default, [email-masked] is a generic mail address. It can accept mails from both human and websites.  A user have to enable a setting called "Restricted Mode" to instruct the system that it's a human-to-human mail address. We heavily rely on MX record instead of SPF record to detect mail genuinity in human-to-human mails.
> 
> On Wed, Sep 25, 2019 at 1:37 PM Valdis Klētnieks <[email-masked]> wrote:

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 25 Sep 2019 13:25:36 +0200

On Wed 25/Sep/2019 12:29:44 +0200 Viruthagiri Thirumavalavan wrote:
> [email-masked] <mailto:[email-masked]>, our system going to fetch the MX
> record and check whether the mail is really coming from example.com

MX /receive/ mail, mailout hosts may differ and, in large sites, they typically do.

> Since we are talking about human-to-human mails here, we are expecting the
> mail from one of your MX servers. We also check SPF record and A record. If
> the mail is not coming from any of those IP addresses, we actually reject
> the mail.

SPF works better.  However, consider the analysis depicted here:
https://en.wikipedia.org/wiki/File:Mailflows-reloaded.png

Many consider reject-on-SPF-fail dubious, which is why most mail sites have
~all instead of -all.  Rejecting on non-pass is definitely bad.  DMARC needs
simultaneous non-pass of both SPF and DKIM in order to reject.  However, the
most diligently authenticated messages are spam.

Best
Ale
--

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 17:05:57 +0530

>
> I print on a business card my address [email-masked] and give it to
> humans.  The humans store the email addresses in a
> database.  The database gets stollen and thus my address [email-masked] gets
> stollen.  Then a “website” sends emails to

This scenario is perfectly fine in my system. The "website" domain's MX,
SPF and A records will be checked. The "website" domain will get
blacklisted if they are abusing the system. This is where C/R becomes
useful.

> Returning challenge emails is nothing new, and is not a solution.  It
> imposes delays, and bothers the sender, just like
> challenge.  In any case, as far as this can be set up per recipient and
> the recipient decides to bother the senders with

C/R mechanism is an optional mechanism in my system. We are not forcing
anyone to enable that. You can pretty much depends on the regular spam
filter here. Since we accept mails only from "verified strangers", it can
cut down a lot of spam.

On Wed, Sep 25, 2019 at 4:15 PM Дилян Палаузов <[email-masked]>
wrote:

> Hello,
>
> I print on a business card my address [email-masked] and give it to
> humans.  The humans store the email addresses in a
> database.  The database gets stollen and thus my address [email-masked]
> gets stollen.  Then a “website” sends emails to
> This scenario is not so rare.  Users use the same username and password
> for different service providers.   Once data
> from any service provider is leaked, which happes regularly, that
> credentials are tried as login on well-known email
> providers and then all contacts stored in the addressbook of that email
> provider get spam, from a mailbox they know (so
> no challenge emails).  Even if the mailbox is not used to send spam, the
> hacker has the [email-masked] address and the
> As I said, first send emails from domains, whose MX records point to your
> own servers, then propose improvements to the
> > When the MAIL FROM says that the mail is coming from [email-masked],
> our system going to fetch the MX record and
> You cannot know, if the sending site is entitled to send emails from
> example.com by looking the MX records.  MX records
> Returning challenge emails is nothing new, and is not a solution.  It
> imposes delays, and bothers the sender, just like
> challenge.  In any case, as far as this can be set up per recipient and
> the recipient decides to bother the senders with
> Another way to slow down spammers, is to return delayed OK on EHLO, RCPT,
> MAIL FROM: and DATA END-DOT commands, with the
> > The system offers different address structures.  We decide how to treat
> the mail based on the RCPT TO address. If it is look like RCPT TO: <
> > By default, [email-masked] is a generic mail address. It can accept
> mails from both human and websites.  A user have to enable a setting called

-- 
Best Regards,

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 17:18:24 +0530

> MX /receive/ mail, mailout hosts may differ and, in large sites, they

Yes, I agree with that. We actually fetch third party MX host SPF record as
well.

I have illustrated that in this image.

https://miro.medium.com/max/684/1*PKHfrnCDFy8-kvaUDY83_Q.png

Also this is how we deal with incoming human-to-human mails when the sender
is an "Unverified Stranger".

mail.example.com Connecting to mail.domboxmail.com with its IP address
domboxmail.com => 220 mail.domboxmail.com Dombox SMTP Service Ready
example.com => HELO mail.example.com
domboxmail.com => 250 Hello, nice to meet you, mail.example.com
example.com => MAIL FROM: <[email-masked]>
domboxmail.com => 250 OK
example.com => RCPT TO: <[email-masked]>
domboxmail.com => 250 OK
example.com => RCPT TO: <[email-masked]>
domboxmail.com =>

*550 Restricted Box. Unauthorized and UnverifiedSender. Please configure
SPF or Send this mail from one of your MXserver IP address*
example.com => RCPT TO: <[email-masked]>
domboxmail.com => 250 OK
example.com => RCPT TO: <[email-masked]>
domboxmail.com =>

*550 Restricted Box. Unauthorized and UnverifiedSender. Please configure
SPF or Send this mail from one of your MXserver IP address*
example.com => RCPT TO: <[email-masked]>
domboxmail.com => 250 OK
example.com => DATA
domboxmail.com => 354 End data with <CRLF>.<CRLF>
{Message Part goes here}
domboxmail.com => 250 OK, message accepted for delivery: queued as 12345
example.com => QUIT
domboxmail.com => 221 Bye

As I mentioned earlier, my system is designed in a way to deal with spam
mails without wasting bandwidth. So all validation mechanism happens before
the DATA command.

-- 
Best Regards,

---

From: Дилян Палаузов <[email-masked]>
Date: Wed, 25 Sep 2019 12:10:46 +0000

Hello,

> domboxmail.com => 550 Restricted Box. Unauthorized and Unverified
> Sender. Please configure SPF or Send this mail from one of your MX
> server IP address
> 

This is not backward compatible and does not work with indirect mail flow.  As an example, for aegee.org no SPF records
will be installed and the MX records will not be adjusted.  It is up for every user to choose which server it will use
to send emails.  The lack of enforced DMARC validates this statement, somehow.  I *guess* for iki.fi the situation is
the same.  The iki.fi provider handles only incoming emails, how users send emails is up to the user (I guess).

So it likely works with most setups, but if users want service that works with all email flows, your system will not be
first choice.  Compare to having a system that does not accept * (asterisk) sign in the local part of the email address
- such host cannot accept all valid emails.

Regards
  Дилян

> As I mentioned earlier, my system is designed in a way to deal with spam mails without wasting bandwidth. So all validation mechanism happens before the DATA command..
>

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 18:10:21 +0530

>
> This is not backward compatible and does not work with indirect mail
> flow.  As an example, for aegee.org no SPF records
> will be installed and the MX records will not be adjusted.  It is up for
> every user to choose which server it will use
> to send emails.  The lack of enforced DMARC validates this statement,
> somehow.  I *guess* for iki.fi the situation is

When I meant backward compatible, I didn't mean all the systems on the
internet are already properly configured to be used with my system. I meant
majority of the cases can work properly with my system and the remaining
abnormal cases can able to easily recognise the problem via our error
message and then ask their administrators to configure the system properly.

>
> Compare to having a system that does not accept * (asterisk) sign in the
> local part of the email address

I guess you are talking about my dollar-based disposable email address
structure here.  We also have a subdomain based email address structure.
Our system can accept mails from both type of addresses. Even if that part
won't work, then we can start to hash the local part. So i'm sure we can
overcome those kind of issues.

I'm not sure whether you have gone through my paper. If you are not, I
would be really grateful if you go through my work and then judge my work.
Because sometimes you need context to understand why the system is designed
in that way. It's really hard to explain everything in a single blog post.

On Wed, Sep 25, 2019 at 5:40 PM Дилян Палаузов <[email-masked]>
wrote:

> This is not backward compatible and does not work with indirect mail
> flow.  As an example, for aegee.org no SPF records
> will be installed and the MX records will not be adjusted.  It is up for
> every user to choose which server it will use
> to send emails.  The lack of enforced DMARC validates this statement,
> somehow.  I *guess* for iki.fi the situation is
> So it likely works with most setups, but if users want service that works
> with all email flows, your system will not be
> first choice.  Compare to having a system that does not accept *
> (asterisk) sign in the local part of the email address
> the DATA command..

-- 
Best Regards,

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 25 Sep 2019 14:50:52 +0200

On Wednesday 25 September 2019 13:35:57 CEST, Viruthagiri Thirumavalavan 
wrote:
> C/R mechanism is an optional mechanism in my system. We are not 
> forcing anyone to enable that. You can pretty much depends on 
> the regular spam filter here. Since we accept mails only from 
> "verified strangers", it can cut down a lot of spam.

I'm curious about how you accept things like mail from a hotel sent via 
sendgrid.com. "Subject: Thanks for booking", that sort of thing. Do you 
expect both the hotel and Sendgrid to adapt to your system? Or do you 
expect users to never give a hotel their ordinary email address? Or do you 
have a novel idea?

Arnt

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 25 Sep 2019 18:48:46 +0530

> expect users to never give a hotel their ordinary email address? Or do you

Our primary validation mechanism focus only on "Envelope Domain". In a
human-to-human mails, both "Envelope From" and "Message From" gonna be same
usually. This is the reason why I'm relying on MX, SPF and A records for
validation.

Page 37 of my white paper actually has an example for sendgrid.

There are three cases here. Let's say the hotel domain is acme.com

Case 1: Isolated Mailbox

You can generate a disposable email address for acme.com like
[email-masked]. In this case acme.com has to whitelist
sendgrid.net via their SAD record to deliver mail. _sad.acme.com => "v=sad1
sendgrid.net -all"

If the domain is not found in the SAD record, then the sender will get
error message like "550 Alias Layer Failure. Whitelist the domain in your
SAD record"

Case 2: Normal Mailbox - Unrestricted

In this case, you have not enabled the "restricted mode" which is the
default mode. So you are saying the mail can come from anywhere. It can be
from human, website etc. So we accept mail just like a regular mail system
and use spam filter for scanning mails.

Case 2: Normal Mailbox - Restricted

In this case you are actually saying, you are gonna use the mailbox only
for "human-to-human" mails. If you actually give out your normal email
address to the hotel acme.com, the mail will be accepted as long as
sendgrid.net configured the SPF. Or they will get an error message like
"550 Restricted Box. Unauthorized and Unverified Sender. Please configure
SPF or Send this mail from one of your MX server IP address".

Also note, the "restricted mode" is specially designed for the
human-to-human mails. If you give out your normal mail address to hotel
sites like acme.com, then C/R mechanism won't be helpful here. Because we
actually send the challenge mail to the "Envelope From" address.
sendgrid.net can't able to respond to the challenge mail in this case. So
just stick with the spam filter method.

In restricted mode, we also check the address book. So if the "Envelope
From" address is found in the address book, then the mail will be accepted
even if the sender is an "unverified stranger"

-- 
Best Regards,

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Wed, 25 Sep 2019 14:27:33 -0400

> "Restricted Mode" to instruct the system that it's a human-to-human mail

I'm really interested in seeing how you explain to a user how to turn on
"restricted mode" so they can get emails from their Aunt Sally regarding next
week's family reunion picnic - and have them actually understand it.

You also get to explain to the user how to tactfully apologize to Aunt Sally
for not being at last week's reunion because they weren't expecting the mail to
be from Sally, but from her sister Susan, so Sally wasn't whitelisted.

Remember - from the user's point of view, your email software made them miss
an important mail.

Bonus points for explaining to the user that if Sally's husband Uncle Fred does
a "reply all" to that mail, Fred is going to get a C/R response from your user
and your user will probably never actually see Fred's reply.  Oh, and your user
had whitelisted Fred's *old* address, but didn't know Fred had a *new* address?
Yeah, you can apologize to the user for that problem too....

Double bonus points for explaining how you prevent the spammers from automating
the C/R response using procmail or similar.

Triple bonus points for explaining how this scales, paying particular attention
to help desk issues regarding "lost" emails because a user forgot to enable
restricted mode.

> genuinity in human-to-human mails.

So... instead of looking at a DNS record that tells you where legitimate emall
should be coming from, you're going to look at a record that is almost guaranteed
to not point at a legitimate source for the email.

Yeah, that's going to work *really* well.

I suggest you look at the Received: lines on your copy of this e-mail, and for each step,
figure out what the RFC821 MAIL FROM probably was, the RFC822 From: (which
you don't see until you've accepted the DATA step), and what checking the MX and SPF
entries would have shown for each step along the way.

Take your time - we'll wait.

Please tell us - how much experience do you have with actually running large
scale email systems?  Can you tell us how much email you've processed through
your zero-spam system, and the current rates of false positives and false
negatives?

---

From: Hector Santos <[email-masked]>
Date: Wed, 25 Sep 2019 16:05:04 -0400

It appears to be an application level MUA product with filtering rules.

Quadruple bonus points for extracting an IETF protocol for a common 
interoperability and that functions deterministically, dynamically 
with "No Batteries Required."

-- 
HLS

---

From: "Kurt Andersen (b)" <[email-masked]>
Date: Wed, 25 Sep 2019 21:17:59 +0100

> It appears to be an application level MUA product with filtering rules.

Which is why I'm wondering what it has to do with the protocol-centric
purpose of this list...

--Kurt

---

From: Ned Freed <[email-masked]>
Date: Wed, 25 Sep 2019 12:57:49 -0700 (PDT)

> Remember - from the user's point of view, your email software made them miss

That's assuming you even get this far. We didn't.

We designed and, I regret to say, implemented a big chunk of a system of this
general type about 15 years ago. We called it "autosecretary", and based it on
subaddresses, although that's just a detail.

Our hope was to appeal to some sort of power user who actually was able
to navigate these waters. But add to the problem list the fact that you
can't control the email client, especially for power users, and now you're
stuck trying to control it from an ancillary web interface. And good
luck with that.

				Ned

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 26 Sep 2019 08:13:32 +0530

> Remember - from the user's point of view, your email software made them

"Restricted Mode" isn't the default mode. A user have to turn it on by
clicking a button. A user will get a warning message like this when they
click the "Enable Restricted Mode" button.

>
> Caution:
> You are about to enter a sensitive zone.
> “Restricted Mode” is intended for the boxes that deals with only
> conversational mails.
> So offload all website related mails to the Domboxes before you enable
> this mode.
> When the Restricted Mode is ON, we will send a challenge mail to the
> Sender if the
> sender is not found in your “Address Book”.
> Real users can respond to those challenges. e.g. CAPTCHA. But automated
> and bulk
> mailers cannot. So their mails never gonna reach your inbox when the box
> is Restricted.
> Do you understand what you are signing up for?
> (a) Yes, I know what I’m doing
> (b) No, Get me out of here.

So if the "restricted mode" not enabled, [email-masked] can receive mails
from Aunt Sally, Sister Susan and Uncle Fred without any issues just like
the current mail system.

Aunt Sally, Sister Susan and Uncle Fred gonna send their mails from a valid
mailbox. The whitelist part is necessary only if they are gonna send their
mails in an abnormal manner. In other words, the whitelist is necessary
only for "Unverified Strangers".

C/R mechanism is optional too. Without C/R, the system gonna work just like
the current mail system with spam filter, except mails only from "verified
strangers" are allowed.

> Yeah, you can apologize to the user for that problem too....

We have method like "Intro via Mutual contact" and "Chain of Trust" to
overcome this issue.  I have explained that in my paper
<https://www.dombox.org/dombox.pdf> on page 217 with illustration.

> the C/R response using procmail or similar.

Since we accept mails only from "verified strangers" a spammer need a valid
domain to send out spam. The means, the mail is REALLY coming from the said
domain since the domain is either whitelisted the IP address or mail
received from one of their MX servers. So all spamming domains will be
considered as "verified spammers". We have a blacklist for "verified
spammers". We also check the "domain registration date" to rate limit
mails.

> to help desk issues regarding "lost" emails because a user forgot to enable

You misunderstood here. If the user forgot to enable "restricted mode", the
system will accept all mails.

> entries would have shown for each step along the way.

I did cover the mailing list aspect in page 265 of my white paper.

>
> In a mailing list, there are usually thousands of subscribers like you.
> There will be an
> address for posting the message.
> Let’s say, [email-masked] is the mailing list post address. When
> you post a message, listserver.com forwards the message to all the
> subscribers.
> For example, when the user Giri post a message, the message would look
> like this when
> viewed by others.
> Envelope From: [email-masked]
> Message From: [email-masked]
> The point here is that the “Message From” domain can be any of those 332
> million domains.
> But “Envelope From” domain is gonna be listserver.com

On Wed, Sep 25, 2019 at 11:57 PM Valdis Klētnieks <[email-masked]>
wrote:

> mails

-- 
Best Regards,

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 26 Sep 2019 08:18:28 +0530

I posted my work here to gain some credibility. My apologies if my post
made you guys uncomfortable in any way.

-- 
Best Regards,

---

From: Dave Crocker <[email-masked]>
Date: Wed, 25 Sep 2019 19:56:38 -0700

> made you guys uncomfortable in any way.

The only discomfort is with having to repeat basic, educational material 
for an approach that has been tested and reviewed long ago.  There's 
quite a bit of documentation about prior efforts, and it's nicely 
summarized in:

     http://craphound.com/spamsolutions.txt

What you might not be aware of is that one form or another of proposal 
for ending spam is produced very few years, for the last 30 years.  None 
has proved viable.  That's what makes the summary checklist legitimate.

The larger concern you might consider is your own resistance to feedback 
that is entirely consistent, from a range of highly experienced subject 
matter experts.

d/

-- 
Dave Crocker
Brandenburg InternetWorking
bbiw.net

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 26 Sep 2019 09:01:45 +0530

Hi Dave,

I can clearly see many of these folks trying to help me. I'm not trying to
neglect their comments in any way. I think my response brings such tone.
My intention of explaining those parts is that "I have thought about that
part". Nothing more. And because most of you folks don't have enough time
to go through my 300 page paper, I'm just hoping explaining some parts
would be helpful.

For example, you have linked to this material
<https://craphound.com/spamsolutions.txt>. I could either pretend that I
don't know anything about that material or I could say I'm aware of that
material. If I go with the latter, then that's gonna look like I'm acting
like I know everything and dismissing your opinion. But that's definitely
not my intention. Page 66 of my paper says "A system would fail when it
expects immediate total cooperation from everybody at once.". That part is
actually coming from this original slashdot post
<https://slashdot.org/comments.pl?sid=98024&cid=8373855>. I did enough
research and understood the problem before designing my system.

Most of you in this mailing list are far more experienced and knowledgeable
than I am. I'm totally grateful for your feedback. But as I said earlier,
email spam solutions are very controversial subject.  Just because people
failed in the past 30 years doesn't mean all new solutions have no merits.
Failure is a stepping-stone and it ought to be encouraged. People shouldn’t
be ashamed of their failures and they shouldn’t be afraid to present a
failed solution. Learning from failures is what makes us human. When
someone points out to a particular tree and say “Hey, don’t eat the fruit
from that tree. It’s poisonous”, they are actually saying someone or
something died by eating that fruit. If you traceback to the origin,
there’s always a “subject zero” who gave his/her/its life to give birth to
such a discovery. This discovery is a meaningful “data point” for mankind's
survival. An archive of many such failures is what’s converted into “common
sense”. Without those data points, the average human lifespan will be very
short. So meaningful failures are essential for new discoveries.

My white paper is a result of plenty of "data points" I gathered from other
people failures. Just because I don't have experience with large scale mail
systems, doesn't mean I don't have enough IQ to put together pieces from
other people failures. When I present my solution here, there is higher
possibility it could become an utter failure. But ideas begets ideas. I'm
sure people who cite my work in the future may bring different
perspectives. So my intention here is to bring some exposure to my work. My
apologies if I offended anyone here.

-- 
Best Regards,

---

From: David MacQuigg <[email-masked]>
Date: Wed, 25 Sep 2019 23:57:05 -0700

> that is entirely consistent, from a range of highly experienced subject
The big problem I see with new solutions to the spam problem is - there is
no longer a problem.  Most of the big email services are doing such a good
job of filtering the spam that there is really no incentive to deploy a
whole new system.  I first got interested in this topic in 2003, when spam
was threatening to obliterate email.  I came up with what I thought was an
obvious solution, ran into a solid wall of group-think, and decided to put
it together myself (box67.com).  I ran that service for a few years for
friends and family, and it was very effective - hardly any spam, no lost
emails, and very little effort at maintaining a transmitter reputation
list.  I also had a gmail address, and found myself using that instead of
my box67 address.  At one point, I seriously considered hiring staff to set
up a real email service with customer support and all the necessary
administrative overhead.  box67 is still running, if anyone is interested
in taking over.  I have not updated the reputation database in 15 years, so
the mail flow is now near 100% spam.

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 26 Sep 2019 09:32:58 +0200

On Thu 26/Sep/2019 05:31:45 +0200 Viruthagiri Thirumavalavan wrote:
> <https://craphound.com/spamsolutions.txt>.
> [...]> Failure is a stepping-stone and it ought to be encouraged. People shouldn’t
be> ashamed of their failures and they shouldn’t be afraid to present a failed>
solution.

An approach somewhat more constructive than the above link is John's old wiki:
https://wiki.asrg.sp.am/wiki

If you have a new technique, a concise, crispy description of just how it
differs from the ones already described would be more acceptable than a
300-page whitepaper.

HTH
Ale
--

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 26 Sep 2019 15:33:14 +0530

> https://wiki.asrg.sp.am/wiki

Thanks for linking to that page.

The page says "Many people have invented many anti-spam techniques over the
past decade. And a lot of the techniques keep being reinvented. Our goal
here is to list all the anti-spam techniques we know, both the good ones
and the bad ones.".

I think this is an excellent resource which I have never seen before. The
page doesn't mock anyone. John Levine did an excellent job in documenting
that wiki. He even documented info like Date of first use, Effectiveness,
Popularity, Difficulty of implementation, Where implemented, Harm etc.  I
think this is the kind of resource every spam researcher should read.

> differs from the ones already described would be more acceptable than a

I did publish a summary of my work here
<https://medium.com/@Viruthagiri/dombox-the-zero-spam-mail-system-2b08ff7432cd>.
I had to omit lot of stuffs to even write that blog post. For example, I
didn't write much about stuffs related to "Sign In With Apple" button.
"Sign In With Apple" button is inspired from my "Chapter 10: Teleport" and
"Chapter 11: Data". My paper even contains an idea for "One-Click
Newsletter subscription service" which is similar to "Facebook Like" button
you see on websites but for "email newsletters" [Chapter 12: Telescribe].
And my paper brings an idea called "Mail Score" to bring transparency
between the sender and receiver [Chapter 7: Mail Score]. So my white paper
is a collection of ideas. The medium blog post summary only focuses on the
broader level overview of my system.

The idea of providing my white paper along with the summary is that, I'm
just hoping at least one person who read my medium blog post summary would
also go through my paper till the end and the leave an unbiased opinion
about my work.

-- 
Best Regards,

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Sep 2019 07:25:01 -0400

On 9/26/19 2:57 AM, David MacQuigg wrote:

> The big problem I see with new solutions to the spam problem is - 
> there is no longer a problem..  Most of the big email services are 
> doing such a good job of filtering the spam that there is really no 
> incentive to deploy a whole new system.

Except that spam is still a problem in many ways:  If you want effective 
spam filtering you pretty much have to use a "big email service" or pay 
a lot of money for an appliance.   The bandwidth overhead of spam 
relative to ham is huge, so if you host email yourself you are paying 
for a fatter pipe than you would otherwise need.   (The bandwidth 
overhead of extra headers to aid in blocking spam also seems to be 
significant.)   The concentration of all of that email to a small set of 
"big email service providers" does tremendous harm to users' privacy.   
In some sense this has also made email less interoperable as the "big" 
providers command enough market share to implement their own proprietary 
features.   And the spam filters block too much legitimate mail, thus 
making email a less valuable service.   While email is still widely 
used, the distaste caused by spam and spam filters has pushed users to 
alternative messaging services that are in general much less functional 
overall and much more significant privacy threats, and which have other 
problems besides.

Keith

---

From: Rich Kulawiec <[email-masked]>
Date: Thu, 26 Sep 2019 08:43:45 -0400

On Thu, Sep 26, 2019 at 07:25:01AM -0400, Keith Moore wrote:
> Except that spam is still a problem in many ways:?? If you want effective
> spam filtering you pretty much have to use a "big email service" or pay a
> lot of money for an appliance. 

Not true.  There are a lot of defensive techniques that are simple,
inexpensive, have great FP/FN ratios, are low-bandwidth, are computation
and memory-efficient, maintainable/tunable, and scalable.

Don't believe the hype from the appliance vendors -- all they've done
is take all this stuff (and the open-source software that implements it)
and throw it in a box with a point-and-drool GUI for the incompetent
lusers who can't work at the command line.

---rsk

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Sep 2019 06:24:24 -0700

> lusers who can't work at the command line.

I operated my own mail server for many years—since the late eighties.  A couple of years ago, I got serious about spam mitigation.  After spending countless hours trying to implement solutions, I realized that I was losing the battle, and I finally just switched to using GSuite.

So if what you are saying is true, it’s news to me, and doesn’t match my experience.   The fact that there are open source solutions out there that do a pretty good job is not an indication that the problem is well solved.   My experience is pretty much the same as Keith’s: spam continues to cause serious damage and is actually advantageous for large email providers, but creates a new set of problems because of that.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 26 Sep 2019 06:39:49 -0700

On 9/25/2019 11:57 PM, David MacQuigg wrote:
> The big problem I see with new solutions to the spam problem is - there 
> is no longer a problem.  Most of the big email services are doing such a 
> good job of filtering the spam that there is really no incentive to 
> deploy a whole new system.

This is a false sense of security.

90-95% of the email traffic across the open Internet is spam.  Worse, 
the bad actors are intelligent, aggressive and adaptable.  The current 
situation is a constant arms race.  That defines a fundamentally 
unstable situation, no matter how well the defenders are doing at any 
given moment.

So, yeah, end users typically see only a tiny fraction of spam, but a) 
that requires massive amounts of continuing effort by those running 
filtering agents, and b) enough still gets through to cause real-world 
problems for end users.

d/

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Sep 2019 13:59:31 -0400

I have  to agree with Dave and Keith.  This is an arms race, it
is driving the costs of email up, increasing concentration of
actors and raising privacy risks, and driving diversity down.
Spam-carried malware and phishing attacks continue to do real
damage including leading identity theft.

In an odd way, all of that mostly-effective filtering has the
side effect that there isn't enough spam getting through: the
only way to actually stop the arms race is to treat spamming as
a seriously anti-social activity and spammers as criminals and
parts of criminal enterprises.   As long as legislators and
regulators see very little spam themselves, efforts to get
effective laws and enforcement actions are likely to be
ineffective, especially while self-defined "legitimate email
marketers" continue to press for weak laws and regulations to be
sure their activities are not constrained.   From that point of
view, if we really wanted to stop spam, there is one thing we
could do that might be effective and that has not been tried:
we could try to organize an international Spam Impact day (or
week) and persuade everyone to shut down their filters for that
period.  I don't think such persuasion would be likely to
succeed but, if it did, I think it is safe to assume that many
actors in the political arena would decide it was time to Do
Something.

    john

--On Thursday, September 26, 2019 06:39 -0700 Dave Crocker
<[email-masked]> wrote:

>> The big problem I see with new solutions to the spam problem
>> is - there  is no longer a problem.  Most of the big email
>> services are doing such a  good job of filtering the spam
>> that there is really no incentive to  deploy a whole new
>> system.
> 
> 
> 90-95% of the email traffic across the open Internet is spam.
> Worse, the bad actors are intelligent, aggressive and
> adaptable.  The current situation is a constant arms race.
> That defines a fundamentally unstable situation, no matter how
> well the defenders are doing at any given moment.
> 
> So, yeah, end users typically see only a tiny fraction of
> spam, but a) that requires massive amounts of continuing
> effort by those running filtering agents, and b) enough still
> gets through to cause real-world problems for end users.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 26 Sep 2019 11:12:14 -0700

On 9/26/2019 11:01 AM, David MacQuigg wrote:
> The volume of spam is not a problem for a properly implemented Receiver,

There is an unfortunate and fundamental error in your choice of vocabulary.

For recipients using highly competent email providers, the volume of 
spam is not a problem for the recipient.

But for those providers, and all others, spam continues to be a very 
serious problem.  (cf, m3aawg.org and apwg.org.)

d/

---

From: David MacQuigg <[email-masked]>
Date: Thu, 26 Sep 2019 11:01:26 -0700

The volume of spam is not a problem for a properly implemented Receiver,
like I had at box67.com http://open-mail.org/BorderPatrol.html   As for the
super-human intelligence of spammers, I have not seen any scheme that can
defeat a well-implemented whitelist.

> filtering agents, and b) enough still gets through to cause real-world

I was surprised by how little effort it took to whitelist a few dozen of
the biggest senders, which accounted for maybe 95% of our legitimate
incoming mail.  That's a 20X improvement over sending it all through a
statistical filter.  Yes, there is still some doubt whether we could have
"scaled up" to a system the size of any of the big services, but the
possibility of scaling up was always in the design.  Before I lost
interest, and moved on to other projects, we had plans to use our Receiver
at the University of Arizona with some 74,000 recipients.  This would have
required 2 or 3 students working part-time to maintain our whitelist.  I am
confident it would have worked.  We just couldn't get any funding.  Also, U
of A decided to let Google handle all their mail.

As for the continuing problems of real-world users, I suggest that they
switch to a service like gmail or yahoo.

---

From: Ted Lemon <[email-masked]>
Date: Thu, 26 Sep 2019 11:50:55 -0700

On Sep 26, 2019, at 11:01 AM, David MacQuigg <[email-masked]> wrote:
> I was surprised by how little effort it took to whitelist a few dozen of the biggest senders, 

Yes.  IOW, the problem of spam naturally selects for large email providers.

---

From: John C Klensin <[email-masked]>
Date: Thu, 26 Sep 2019 18:43:03 -0400

--On Thursday, September 26, 2019 16:22 -0400 John Leslie
<[email-masked]> wrote:

> John C Klensin <[email-masked]> wrote:
>> 
>> I have  to agree with Dave and Keith.  This is an arms race,
>> it is driving the costs of email up, increasing concentration
>> of actors and raising privacy risks, and driving diversity
>> down. Spam-carried malware and phishing attacks continue to
>> do real damage including leading identity theft.
> 
>    +++++
> 
>> only way to actually stop the arms race is to treat spamming
>> as a seriously anti-social activity and spammers as criminals
>> and parts of criminal enterprises.
> 
>    Umm... I disagree.
> 
>    Spam is not a contry-by-country problem. It is world-wide.

And I didn't say a word about country-by-country anything.  And,
if countries were to line up to solve the problem, I'd expect
the US Congress to be close to the end of the line, right in
front of those countries whom some of us suspect of having
anti-spam policies that, in practice, amount to "attack anyone
you like as long as you don't attack other citizens of our/your
country".  Because of that world-wide problem, one that from my
perspective involves spammers hiding in one country to escape
the jurisdiction of others, a considerable amount of
international cooperation is obviously necessary.  However,
there are precedents for that which go back several centuries
and they have a good success record... if the problem is
perceived as serious enough.

>    We are better able to address world-wide problem than any
> individual contry's legislature (least of all the US
> Congress!).

What you mean "we"?  You can remember the days when there were
widely-held suspicions that anti-virus vendors were suspected of
spreading viruses to increase the market for their products.  I
haven't heard of those suspicions being leveled against any of
the anti-spam vendors and operators.  However, at least some of
them have incentives to let the spam level remain high enough
that they keep selling products and services, even if none of
them are motivated by it.  Similarly, the very large providers
have incentives to see more and more email users driven to their
platforms (and to view the need to interoperate with smaller
providers as an inconvenience) even if, again, one does not
believe that drives their behavior.   

If you consider the present situation, after all these years of
spam being widely considered as a scourge or worse, as
successfully addressing the problem, we disagree about the
definition of "success".

>> As long as legislators and regulators see very little spam
>> themselves, efforts to get effective laws and enforcement
>> actions are likely to be ineffective,
> 
>    For US legislators, at least, _none_ of them deal with spam
> as part of their regular email process. They have staffers
> which _may_ read an actual email account: the legislator
> couldn't possibly read all their "ham" from
> probable-constituents.

Yes.  But that was part of my point.  And, again, the US is not
at the top of my list of concerns.

>> especially while self-defined "legitimate email marketers"
>> continue to press for weak laws and regulations to be sure
>> their activities are not constrained.
> 
>    This is true. (and the "email marketers" contribute to
> re-election campaign funds, while spam-receivers mostly don't).

Indeed.
 
>> From that point of view, if we really wanted to stop spam,
>> there is one thing we could do that might be effective and
>> that has not been tried: we could try to organize an
>> international Spam Impact day (or week) and persuade everyone
>> to shut down their filters for that period.
> 
>    An interesting idea!!
 
>> I don't think such persuasion would be likely to succeed but,
>> if it did, I think it is safe to assume that many actors in
>> the political arena would decide it was time to Do Something.
 
>    Even if they did, how could enough legislators agree what
> to do???

Depends on how much public (or constituent or donor) pressure
they were feeling.  An equally good question, however, is
whether we would have a reasonable expectation that, if they
agreed on something, it would be realistic, plausible, and would
not cause more problems than it solved or whether they would
come up with bad solutions that would have to do more damage
before iteration was possible.  I have no confidence about that
at all.  On the other hand, that takes me back to where we
disagree -- I think our current path and mechanisms are an
increasingly expensive dead end, unless it results in a very
small number of providers who are willing to authenticate and
police the behavior of their own users and establish trust
relationships with each other.  I don't consider that a
desirable outcome and that is part of what puts me in some
alignment with Dave and Keith, but YMMD and I'm confident that
of most of those operates does.

>    BTW, I carefully read the entire 300-page document before
> replying. There are interesting ideas there -- not that we
> could simply standardize this document -- but pieces for which
> we could standardize an underlying structure. I'd be happy to
> work on something of that order. (I have had no luck so far
> getting a consensus to work on anything like that.)

I have not studied that much of it, but share your impression
that there are some interesting ideas there.  For me, whether
they are interesting enough to overcome the difficulties others
have pointed out, including the formidable barriers to
widespread deployment and some trust issues, is another
question, but one that I have not studied the proposal
sufficiently to comment on (and I therefore haven't)

best,
    john

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Thu, 26 Sep 2019 18:52:18 -0400

On Thu, 26 Sep 2019 13:59:31 -0400, John C Klensin said:
> parts of criminal enterprises.

I tend to phrase it as "We're not going to get rid of e-mail spam until
the solution involves a horse, a rope, and a high branch...."

One problem is that a lot of the spammers are in jurisdictions that
remind me of a Monty Python skit:

Vercotti: (....) and one evening in walks Dinsdale with a couple of big lads,
one of whom was carrying a tactical nuclear missile. They said I had bought one
of their fruit machines and would I pay for it.
2nd Interviewer: How much did they want?
Vercotti: They wanted three quarters of a million pounds.
2nd Interviewer: Why didn't you call the police?
Vercotti: Well I had noticed that the lad with the thermonuclear device was the
chief constable for the area.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 26 Sep 2019 17:25:44 -0700

> the solution involves a horse, a rope, and a high branch...."

That is overly optimistic.  After all, we've demonstrated that it 
doesn't stop other crimes, so why would it work here?

My comment is meant both for the humor (hoping it is shared) and for the 
reality check.  The model needs to be for reduction, not elimination.

d/

---

From: Keith Moore <[email-masked]>
Date: Thu, 26 Sep 2019 20:39:15 -0400

> doesn't stop other crimes, so why would it work here?

Deterrence works better for some kinds of crimes than others. When the 
criminal is motivated by financial gain, deterrence can work better than 
if the criminal is motivated by passion or ideology.

(But of course you're right that it's overly optimistic. Clearly we need 
a lot more than one horse, one rope, and one tree...)

Seriously, for deterrence to produce a significant reduction in spam, 
punishment for spamming is going to have to become a lot more certain 
than it is now.   Until it becomes normal for spammers to be prosecuted, 
and to pay significant fines for first or relatively minor offenses, and 
serve prison terms for repeat offenses, we're going to keep seeing it.

Keith

---

From: David MacQuigg <[email-masked]>
Date: Thu, 26 Sep 2019 18:13:49 -0700

> There is an unfortunate and fundamental error in your choice of vocabulary.

I don't have time to argue over vocabulary.  I'll just say that I find it
much more convenient to say "Receiver" instead of "the first Agent on the
Recipient's side of the Border, the one that interfaces directly with the
Transmitter".  See http://en.citizendium.org/wiki/Email_system for
definitions of these terms and examples of how they are useful in
describing real systems.

---

From: Dave Crocker <[email-masked]>
Date: Thu, 26 Sep 2019 18:18:01 -0700

On 9/26/2019 6:13 PM, David MacQuigg wrote:
> I don't have time to argue over vocabulary.  I'll just say that I find 
> it much more convenient to say "Receiver" instead of "the first Agent on 
> the Recipient's side of the Border, the one that interfaces directly 
> with the Transmitter".  See 
> http://en.citizendium.org/wiki/Email_system for definitions of these 
> terms and examples of how they are useful in describing real systems.

For some reason, I prefer to used standardized terminology.  It tends to 
have more shared meaning. Otherwise terminology might be carefully 
applied by the speaker but differentially understood by the listener. 
That's the essence of ambiguity.

Hence, RFC 5598 wins the find-a-definition game.

d/

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Thu, 26 Sep 2019 22:40:47 -0400

> definitions of these terms and examples of how they are useful in

Hate to tell you this, Dave is probably quite aware of what "receiver"  and
"recipient" means in an e-mail protocol context, and he's using them correctly.

(Anybody else having a flashback to the time somebody was trying to explain
RIPv2 to Tony Li? :)
 
His point was that it's not a problem for the end *recipient*. It is, in
general, a very real problem that sucks considerable resources and manpower for
*receiver* - "the first Agent on the Recipient's side of the Border".

When your Receiver server farm is spending more time dealing with rejecting spam
than it is in receiving and delivering legitimate mail, it's a problem.

---

From: David MacQuigg <[email-masked]>
Date: Fri, 27 Sep 2019 01:43:03 -0700

> Hence, RFC 5598 wins the find-a-definition game.

RFC 5598 is great for describing email systems at the relay level, but we
need something more for a simple description at the administrative level.
Even the most complex network of relays can be modeled as a simpler network
of Actors, each with well-defined roles in an email system.  Understanding
these roles is critical to understanding spam and email security.

I remember learning about email systems, primarily from RFC 5598.  I found
Section 2 frustrating and had to give up and develop my own model at the
administrative level.  I have found this model very useful in teaching and
in developing my own email system, box67.com.

---

From: Laura Atkins <[email-masked]>
Date: Fri, 27 Sep 2019 10:08:49 +0100

> On 27 Sep 2019, at 01:39, Keith Moore <[email-masked]> wrote:
> 
>> That is overly optimistic.  After all, we've demonstrated that it doesn't stop other crimes, so why would it work here? 
> Deterrence works better for some kinds of crimes than others.   When the criminal is motivated by financial gain, deterrence can work better than if the criminal is motivated by passion or ideology. 
> (But of course you're right that it's overly optimistic.   Clearly we need a lot more than one horse, one rope, and one tree...)
> 
> Seriously, for deterrence to produce a significant reduction in spam, punishment for spamming is going to have to become a lot more certain than it is now.   Until it becomes normal for spammers to be prosecuted, and to pay significant fines for first or relatively minor offenses, and serve prison terms for repeat offenses, we're going to keep seeing it.  
> 
And that has worked so well for other non-violent crimes like … speeding and drug possession. 

Good luck with that. 

laura 

-- 
Having an Email Crisis?  We can help! 800 823-9674 

Laura Atkins
Word to the Wise
[email-masked]
(650) 437-0741		

Email Delivery Blog: https://wordtothewise.com/blog

---

From: Keith Moore <[email-masked]>
Date: Fri, 27 Sep 2019 08:35:38 -0400

On 9/27/19 5:08 AM, Laura Atkins wrote:

>> than it is now.   Until it becomes normal for spammers to be 
>> prosecuted, and to pay significant fines for first or relatively 
>> minor offenses, and serve prison terms for repeat offenses, we're 
>> going to keep seeing it.
>>

Not valid analogies.   Part of the problem with reducing speeding and 
drug possession is that so many of the people are doing them.   If 
everyone were spamming, it would indeed be hard to police it.   But that 
doesn't seem to be the case.   The problem with policing spamming is 
different - it's too easy for spammers to hide their tracks, e.g. with a 
botnet.

Keith

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 27 Sep 2019 15:57:03 +0200

On Friday 27 September 2019 14:35:38 CEST, Keith Moore wrote:
> Not valid analogies.   Part of the problem with reducing 
> speeding and drug possession is that so many of the people are 
> doing them.   If everyone were spamming, it would indeed be hard 
> to police it.   But that doesn't seem to be the case.   The 
> problem with policing spamming is different - it's too easy for 
> spammers to hide their tracks, e.g. with a botnet.

Lots don't.

If we got spam _reporting_ up to the same level of technical proficiency as 
we carry out _filtering_, then the problem _might_ be reduced 
significantly.

I don't think Sendgrid knows that it's spamming me for a get-rich-quick 
scheme or Tinyurl that the same spammer uses it for his landing pages, I 
don't think Rapidswitch (a small rent-a-server place) knows that a customer 
is trying to swindle me (latest iphone 3€!!!).

So I a sunny optimist and believe that sendgrind, tinyurl and most hosters 
mean well. I do wish I had some practical suggestions for how to make them 
be well-informed as well as well-meaning.

Arnt

---

From: Dave Crocker <[email-masked]>
Date: Fri, 27 Sep 2019 07:15:24 -0700

On 9/27/2019 1:43 AM, David MacQuigg wrote:
> I remember learning about email systems, primarily from RFC 5598.  I 
> found Section 2 frustrating and had to give up and develop my own model 
> at the administrative level.  I have found this model very useful in 
> teaching and in developing my own email system, box67.com 
> <http://box67.com/>.

1. If you only found that one section frustrating, you are doing better 
than most others, including me.(*)

2. The thing about RFC 5598 is that it was developed in a consensus 
process.  So whatever its faults, it represents a meaningful degree of 
community compromise.  For this topic, no other equivalent document can 
say that.

3. Pursuing discussion of improvements to RFC 5598 would be entirely 
reasonable.

4. (This adapts from my original note)  Using language that has no 
shared understanding prevents communication. If one wants to make a 
point with other folks, one needs to use language that is well 
understood by the target audience.

5.  Your text, quoted above, cites an alternative bit of work, but does 
not provide a citation to it.  Instead there is a generic citation to 
work that 'uses' it, apparently.  Following that link produces a page 
that starts with "Much of what is described in these pages is not yet 
implemented."  So to the extent you meant to give us something superior 
or even useful, that isn't the way to do it.

d/

(*) Computer networking got real in the 1970s.  Lots of buzz in the 
technical community, well-motivated by the public demonstration of the 
Arpanet (but also by other work.) I got involved in 1972, as a college 
dropout, knowing nothing about the work or much of anything about 
computer science.  I noted that when people from different teams met, 
the first half of the conversation would define terms and the second 
half would use them.  The OSI 7-layer model was issued in the mid 1970s 
and it changed things profoundly.  When people from different teams met, 
they would spend the first half of the conversation criticizing the OSI 
model.  They would use the second half of the conversation using it.

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 27 Sep 2019 17:53:25 +0200

On Fri 27/Sep/2019 15:57:03 +0200 Arnt Gulbrandsen wrote:
>> Not valid analogies.   Part of the problem with reducing speeding
>> and drug possession is that so many of the people are doing them.  
>> If everyone were spamming, it would indeed be hard to police it.  
>> But that doesn't seem to be the case.   The problem with policing
>> spamming is different - it's too easy for spammers to hide their
>> tracks, e.g. with a botnet.
> 
> Lots don't.
> 
> If we got spam _reporting_ up to the same level of technical
> proficiency as we carry out _filtering_, then the problem _might_ be
> reduced significantly.

There is also an inordinate amount of password cracking and similar
bot-like activities which can be reported...  A few ISPs seem to react
adequately to reports of such abuses.

For analogies, how about pollution?

With minor tweaks, SMTP could support well CPUs slowing down so as to
cut energy consumption.  The Internet reportedly contributes some 2~3%
of carbon emissions.  However, spammers make such kind of energy
saving attempts out of the question.  Even on a tiny mail server, I
never need to ping to check if the network is up, it's enough to tail
-f the mail log.

Spam is like waste abandoned on the side of the road, or burned in the
backyard.  It forces servers to consume more energy than they would
otherwise.  It pushes climate crisis.

Best
Ale
--

---

From: David MacQuigg <[email-masked]>
Date: Fri, 27 Sep 2019 08:53:36 -0700

> than most others, including me.(*)

Compared to other RFCs, I would say 5598 is excellent.

> community compromise.  For this topic, no other equivalent document can

Agree.

> 3. Pursuing discussion of improvements to RFC 5598 would be entirely

Wow!!  My first thought is "let's jump right in".  But then I think - no, I
don't have time for this.  I haven't been working with email systems for
over ten years, and my main focus right now is managing a horse ranch.

> understood by the target audience.

Perhaps we could view an elaboration of the Administrative Level as an
extension of what is in RFC 5598.  We need better definition of the
specific roles of the Actors, not just call them all ADMDs.  We need
something equivalent to Fig. 1 in
http://en.citizendium.org/wiki/Email_system

> or even useful, that isn't the way to do it.

See the Citizendium link above for a general introduction.  See the links
at box67.com for details on the system I put together.  This is a
whitelisting system based on the reputation of the Transmitter's domain
name.  It uses DNS to host a database of authentication and reputation
information.  It worked even better than I expected.  Work stopped in 2008
when I started using gmail, and was no longer bothered by spam.  I haven't
kept up with developments in the last ten years, but I suspect gmail and
all the other services that are very good at blocking spam, are using some
form of proprietary reputation system.  That's a shame, because an
open-source system would have the possibility of becoming a de-facto
standard, and that would get us to a zero-spam mail system (at least for
those who would care enough to follow the standard).

---

From: Dave Crocker <[email-masked]>
Date: Fri, 27 Sep 2019 09:32:07 -0700

> http://en.citizendium.org/wiki/Email_system

I see multiple errors in the chart there.

First, citing Browser as an alternative to Client is a misunderstanding 
of the role of a browser in email.  I understand the tendency to claim 
that 'web mail' is somehow distinct, but really it isn't.  It is merely 
a particular approach to the implementation of an MUA, splitting its 
functionality between a server and code /inside/ the browser.

Second is citing client and browser as connecting to the MDA, rather 
than the mailstore, which is what they really interact with.  They do 
this without the involvement of the MDA.

Otherwise, the page's use of the term 'actor' is roughly on a par -- but 
with more limitations -- with RFC 5598's Section 2.1

RFC 5598's other Section 2 uses of the term 'actor' cover significantly 
different contexts than the citizendium use.  For example, 
Administrative role is quite different from user-level role.

d/

---

From: John Leslie <[email-masked]>
Date: Thu, 26 Sep 2019 16:22:29 -0400

+++++

   Umm... I disagree.

   We are better able to address world-wide problem than any
individual contry's legislature (least of all the US Congress!).

> As long as legislators and regulators see very little spam themselves,
> efforts to get effective laws and enforcement actions are likely to be
> ineffective,

   For US legislators, at least, _none_ of them deal with spam as
part of their regular email process. They have staffers which _may_
read an actual email account: the legislator couldn't possibly read
all their "ham" from probable-constituents.

> especially while self-defined "legitimate email marketers" continue
> to press for weak laws and regulations to be sure their activities
> are not constrained.

   This is true. (and the "email marketers" contribute to re-election
campaign funds, while spam-receivers mostly don't).

> From that point of view, if we really wanted to stop spam, there is
> one thing we could do that might be effective and that has not been
> tried: we could try to organize an international Spam Impact day
> (or week) and persuade everyone to shut down their filters for that
> period.

   An interesting idea!!

> I don't think such persuasion would be likely to succeed but, if it
> did, I think it is safe to assume that many actors in the political
> arena would decide it was time to Do Something.

   Even if they did, how could enough legislators agree what to do???

   BTW, I carefully read the entire 300-page document before replying.
There are interesting ideas there -- not that we could simply
standardize this document -- but pieces for which we could standardize
an underlying structure. I'd be happy to work on something of that order.
(I have had no luck so far getting a consensus to work on anything like
that.)

--
John Leslie <[email-masked]>

---

From: Дилян Палаузов <[email-masked]>
Date: Fri, 27 Sep 2019 18:43:17 +0000

Hello,

my opinion on legislation:

Receiving spam and stealing time is the same.  Every country wants to preserve its right to steal time of citizens on
one or another matter and even if the citizen was right eventually on that matter, the stolen time is not reimbursed.

In the EU, the member states (executive, legislative and judicative) establish a body (agency), that collects penalties
from spammers and it is assumed, that this body protects sufficiently the natural and juridical persons from spam.  This
means, when the executive/judicative receives spam, it handles the case to that body, the agency collects penalities
from the spammers and the collected money is used for anti-spam measures.

This means, that the governments protect theirselves sufficiently from spam, by collecting money from the spammers and
using that money to buy anti-spam software, but pretend at the same time, that anti-spam software is not necessary to be
sufficiently protected from spam and stolen time costs no money.

Outrage against society.

Дилян

---

From: S Moonesamy <[email-masked]>
Date: Fri, 27 Sep 2019 14:03:51 -0700

Hi Viruthagiri,
At 07:06 PM 24-09-2019, Viruthagiri Thirumavalavan wrote:
>I would like to present my work here. It's a new but backward 
>compatible mail system. It addresses some of the issues we face in 
>the current email system. E.g. Email Spam, Phishing, Privacy, Data 
>Breach, Bandwidth Wastage etc.

It looks like you put in a lot of work into the paper.  One of the 
proposals in the paper is to have "challenge mails" and there is a 
sentence on Page 228 about it causing inconvenience.  The challenge 
form (Page 229) is even more inconvenient as I would have to decipher 
some of the characters.  That approach has become an accessibility 
challenge nowadays.

How would the system which is being proposed handle the following: 
https://mailarchive.ietf.org/arch/msg/ietf/s_Y8Vux32Ei_0HCBiiVzb-4hhFg ?

Regards,
S. Moonesamy

---

From: David MacQuigg <[email-masked]>
Date: Fri, 27 Sep 2019 15:37:19 -0700

> functionality between a server and code /inside/ the browser.

Good observation.  I'm not clear how to clarify the diagram, however.  What
it is intended to show is that the Recipient can retrieve his email either
through a browser or through some other client software, and the protocols
communicating with the MDA are different.

> this without the involvement of the MDA.

I view the Mailstore as part of the MDA, whether it is actually disks
inside the MDA, or somewhere out in the cloud.  Maybe leaving out the label
"Mailstore" will make the diagram more clear.  I don't see any scenario in
which we need to treat them as separate Agents.  The essential
simplification is that the Recipient can view them as one Mailstore and
Delivery Agent.

> Administrative role is quite different from user-level role.

An Actor in Fig. 1 can be a User or an Agent.  Agents are defined by their
Roles and Responsibilities http://en.citizendium.org/wiki/Email_agents   Agents
could include entities like the Registry, which we might connect with a
dotted line to the Receiver.

Can you create a diagram equivalent to Fig. 1, using only the entities and
terminology in RFC 5598?

---

From: Dave Crocker <[email-masked]>
Date: Fri, 27 Sep 2019 15:50:13 -0700

On 9/27/2019 3:37 PM, David MacQuigg wrote:
> Good observation.  I'm not clear how to clarify the diagram, however.  
> What it is intended to show is that the Recipient can retrieve his email 
> either through a browser or through some other client software, and the 

The browser is merely one of those 'other' client software.  the diagram 
gives it special status.  it's fine to want to show alternatives, but 
they should be equal.

> protocols communicating with the MDA are different.
> 
> I view the Mailstore as part of the MDA, 

It isn't.

I suspect the confusion here is the common one of conflating network 
architecture design with implementation design.  They are quite different.

> whether it is actually disks 
> inside the MDA, or somewhere out in the cloud.  Maybe leaving out the 
> label "Mailstore" will make the diagram more clear.  

It won't.

> Can you create a diagram equivalent to Fig. 1, using only the entities 
> and terminology in RFC 5598?

No doubt one could.

d/

---

From: David MacQuigg <[email-masked]>
Date: Fri, 27 Sep 2019 16:45:19 -0700

> gives it special status.  it's fine to want to show alternatives, but

Which alternatives would you show in this introductory basic diagram?
Should we include the protocols (POP, IMAP, HTTP)?

> > and terminology in RFC 5598?

I can't.  The problem is that every block is just an ADMD, with no
distinction between one and another, no clues to support a discussion of
their different roles and responsibilities, etc.

Imagine you are trying to teach the fundamentals of email systems to
university students in a class on computer networks.  Most are not going to
become email system admins.  Most will be overwhelmed by the level of
detail in Fig. 5 of RFC 5598.  You have one lecture to give these students
a basic understanding of how email systems work.  How would you do it?

---

From: Dave Crocker <[email-masked]>
Date: Fri, 27 Sep 2019 16:57:01 -0700

>      > and terminology in RFC 5598?

If it's a diagram about email, HTTP really has nothing to do with email, 
any more than TCP does.

I suspect the intent is to show the implementation choice of a split UA, 
with http between the MUA front end and the MUA backend.

But these kinds of references tend to fail by leaving out the 
email-specific layer that runs on top of HTTP.  Really, in this use, 
HTTP is nothing but a transport-level function.

> their different roles and responsibilities, etc.

You say you want simplicity but then complain that something is too simple.

> university students in a class on computer networks.  Most are not going 
> to become email system admins.  Most will be overwhelmed by the level of 
> detail in Fig. 5 of RFC 5598.  You have one lecture to give these 
> students a basic understanding of how email systems work.  How would you 
> do it?

I was, indeed, impressed by the level of detail that was required to 
create RFC 5598.

I started the effort knowing it would be require some work, but did not 
anticipate that it would take 5 years and many, many rounds of community 
consultation and community invention.  (The transfer of responsibility 
/inside/ the MSA and MDA was a true revelation to me.)

I also pressed to include only what really was essential to describe the 
/existing/ email service on the Internet.  It kept pissing me off that 
people would raise a concern with the then-current version of the draft 
that was a) not adequately covered by it, and b) could not reasonably be 
deferred.

This was a far cry from the original, simple MUA/MTA distinction we 
created in 1980.

If your requirement is simplicity, note that each of the Figures 1-4 is 
relatively simple.  (The fact that they use ASCII Art makes this a bit 
less obvious.)

The reason for doing multiple figures is to isolate complexity.

While yes, it is good to start class instruction with simplistic models, 
if the classic is to be practical, it needs to get serious students to 
appreciate real-world complexities.

d/

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Fri, 27 Sep 2019 20:02:08 -0400

> simplification is that the Recipient can view them as one Mailstore and

That simplification is OK if you want the reader to understand the view as seen
from a Recipient.

However, if you're actually designing a Mailstore or Delivery Agent
that has to scale to large sizes, you're *really* going to want to have the distinction
clear in your mind.

True story:  My recent employer had a need to set up ITAR-compliant email for
researchers - and one of the requirements is that the data has to reside in servers
on US soil.  This was a problem for GMail, because we couldn't get Google to guarantee
that if one of our researchers was in Europe and Google's location service learned about
it, that it would *not* pre-stage the data in the GMail mailbox to European servers for
faster access if/when the researcher checked their mail.

So in that case, the delivery agent that drops the *first* copy into the mailstore is *very*
different from the mailstore itself which is doing replication and moving from continent to
continent.

We ended up standing up an Exchange server and some NAS storage in a locked room
with more access controls than the rest of our server room.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 28 Sep 2019 09:48:05 +0530

> https://mailarchive.ietf.org/arch/msg/ietf/s_Y8Vux32Ei_0HCBiiVzb-4hhFg ?

Hi Moonesamy,

I think you are referring to accidentally sending challenge mails to
noreply addresses. Please correct me if I'm wrong.

Usually noreply addresses are not falling under human-to-human category. So
we highly recommend our users to offload website related and mailing list
related mails to domboxes before enabling restricted mode. That's because a
dombox address gives exclusive privilege to a domain and its alias domains.

But your concern is a valid concern. I think we should take precautionary
measures for challenge mails. For example, if the MAIL FROM local-part
contains text like "noreply" or "no-reply" and the RCPT TO address requires
CAPTCHA, then we should reject the mails with an error message like "550
Recipient requires CAPTCHA. Not possible in noreply addresses.". We can
also use headers like "List-Unsubscribe" to detect non-human mails.

For the record, I'm not really a fan of challenge mails. But we cannot just
ignore it due to its annoying nature. For example, you have a blog post and
you see thousands of comments posted by bots everyday. You get genuine
comments monthly once. So it's reasonable to enable CAPTCHA here.

Plenty of people in the world use email address only for signing up in
third party websites like Facebook, Youtube etc. They hardly use that for
human-to-human communication. So CAPTCHA makes sense for such folks.

The key takeaway from my work is not the challenge part, it's the "verified
strangers" part. As of now, botnets plays a huge role in email spam. Last
time I checked there are botnets out there that is capable of sending 92
billion spam mails per day. Mirai botnet source code
<https://github.com/jgamblin/Mirai-Source-Code> is available on the github.
So you don't need much technical skills to create a botnet. My system tries
to bring those spammers inside a circle by dividing the system into human
mails and non-human mails. My system tries to punish the domain rather than
IP addresses. So I believe it's effective in dealing with botnet spam.

-- 
Best Regards,

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Sat, 28 Sep 2019 01:41:40 -0400

> Usually noreply addresses are not falling under human-to-human category.

Yes, but how do you identify them?

> also use headers like "List-Unsubscribe" to detect non-human mails.

Using RFC2369 headers to identify non-human mails isn't exactly correct. Hint:
The message you're reading right now has a List-Unsubscribe: header in it, and
it's arguable whether stuff I write is non-human or not. ;)

And assuming that the MAIL FROM contains 'noreply' is a bad choice as well, as
many automated systems will intentionally use 'MAIL FROM:<>' to guarantee that
no bounces come back.

A lot of spam *also* uses <> to ensure there's not bounces coming back. And yes,
some people have tried refusing all mail that comes in with a <> - and discover that
doing so breaks a lot of things.

There's also the Law of Unintended Consequences....

You might want to read up on what happens when Yahoo people post to a mailing
list and the list breaks the very strict DMARC that Yahoo tags on it - the end
result is that systems that get the mail and do a DMARC verify will reject the
mail, send a 550 error back - and the mailing list software will silently
unsubscribe the user from the list.  This is why Mailman has the 'Munge-From'
option for mailing lists.

The end result is that you can end up damaging your own users while trying to
prevent spam.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 28 Sep 2019 11:39:08 +0530

These seem like valid points. I'll take a look at it.

If the challenge mails gonna cause issues, then most likely I will remove
that feature from my system.

Thanks for the input.

-- 
Best Regards,

---

From: John C Klensin <[email-masked]>
Date: Sat, 28 Sep 2019 03:58:50 -0400

--On Friday, September 27, 2019 16:45 -0700 David MacQuigg
<[email-masked]> wrote:

>...
> Imagine you are trying to teach the fundamentals of email
> systems to university students in a class on computer
> networks.  Most are not going to become email system admins.
> Most will be overwhelmed by the level of detail in Fig. 5 of
> RFC 5598.  You have one lecture to give these students a basic
> understanding of how email systems work.  How would you do it?

I'm busy imagining.  I'm imagining that one of the most
important messages I'd want to deliver to such students is that
things are often more complex than they appear at first glance
and/or more complex than one would ideally like them to be.  I'd
also want them to learn that it is important to know the
difference between models that reflect the real world and what
we politely call hand waving.  I'd think about how much easier
it would be to teach TCP/IP if I ignored congestion control and
how much shorter the email talk could be if I could ignore the
implications of gateways to other types of mail systems and
relays, especially relays under different administration/
control than either the user-originator, MSA and associated
systems and the user-recipient, MDA, and associated systems.  I
might then decide that email was not a very good example of an
application in the modern Internet because just about everything
else assumes a direct and real-time originator-recipient
connection.  If I did, I'd wave my hands for maybe 10 or 15
minutes (and make it clear that I was doing that) and then move
on to a more productive example.  Or I might conclude that a
well-developed lesson about growing complexity with changing
needs was really important.  If I did,  I'd start with the model
of the early 1980s, early enough to be well before the DNS and
particularly before RFC 974.  I'd then walk through examples of
how and why the complexity developed until I reached and could
discuss the present reality, including, as Valdis indirectly
points out, distributed and mirrored mailstores with mechanisms
for moving stored messages around to provide better performance
is the user moves from one geographical location to another.  It
that took three lectures rather than one, something else in the
course would need to go.

YMMD, but you asked.

best,
    john

---

From: S Moonesamy <[email-masked]>
Date: Sat, 28 Sep 2019 01:19:48 -0700

Hi Viruthagiri,
At 09:18 PM 27-09-2019, Viruthagiri Thirumavalavan wrote:
>Usually noreply addresses are not falling under human-to-human 
>category. So we highly recommend our users to offload website 
>related and mailing list related mails to domboxes before enabling 
>restricted mode. That's because a dombox address gives exclusive 
>privilege to a domain and its alias domains.

The example which I mentioned in my previous email is an email which 
was authored by a person.  That email was sent through a web page.

>But your concern is a valid concern. I think we should take 
>precautionary measures for challenge mails. For example, if the MAIL 
>FROM local-part contains text like "noreply" or "no-reply" and the 
>RCPT TO address requires CAPTCHA, then we should reject the mails 
>with an error message like "550 Recipient requires CAPTCHA. Not 
>possible in noreply addresses.". We can also use headers like 
>"List-Unsubscribe" to detect non-human mails.

That would cause an email from an IETF reviewer to be rejected.  The 
"List-Unsubscribe" header in this case is not a good signal for 
detecting emails which are not from a person.

>For the record, I'm not really a fan of challenge mails. But we 
>cannot just ignore it due to its annoying nature. For example, you 
>have a blog post and you see thousands of comments posted by bots 
>everyday. You get genuine comments monthly once. So it's reasonable 
>to enable CAPTCHA here.

Ok.

>Plenty of people in the world use email address only for signing up 
>in third party websites like Facebook, Youtube etc. They hardly use 
>that for human-to-human communication. So CAPTCHA makes sense for such folks.

Ok.

>The key takeaway from my work is not the challenge part, it's the 
>"verified strangers" part. As of now, botnets plays a huge role in 
>email spam. Last time I checked there are botnets out there that is 
>capable of sending 92 billion spam mails per day. Mirai botnet 
>source code is available on the github. So you don't need much 
>technical skills to create a botnet. My system tries to bring those 
>spammers inside a circle by dividing the system into human mails and 
>non-human mails. My system tries to punish the domain rather than IP 
>addresse. So I believe it's effective in dealing with botnet spam.

I understand that there is some source code which could be used to 
send a lot of unwanted mail.   According to your proposal (Page 240), 
the "verified strangers" uses a "challenge/response mechanism".  How 
does it prevent those "challenge" emails from being sent to the 
domains used by the botnet?  Will all emails which are not DKIM or 
SPF "authenticated" be discarded?

Regards,
S. Moonesamy

---

From: David MacQuigg <[email-masked]>
Date: Sat, 28 Sep 2019 02:53:00 -0700

On Fri, Sep 27, 2019 at 5:02 PM Valdis Klētnieks <[email-masked]>
wrote:

> label

The User's view (Author or Recipient) is a good starting point for a
simple, informative model.  I've used this model in lectures to students in
a Computer Networks class at U of A, and as a basis for building my own
experimental email system, box67.com.  In class, we never get as far as RFC
5598, but I firmly believe that students are better prepared to learn these
details, if they ever need to later in their careers.

I am not worried about a few necessary departures from the terminology and
models in the RFC.  CS students are used to a boatload of new terms with
each chapter.  What's important at this level is focus on fundamentals and
building a solid foundation of understanding.  I don't let students think
that my presentation is the whole story, especially with complex systems
like email.  I show them Fig. 5 in RFC 5598 to let them know there is lot
more to learn if they ever get serious about email systems.

> that has to scale to large sizes, you're *really* going to want to have

I've never designed an MDA or a Mailstore, but I believe you.  Your example
below is excellent.

> with more access controls than the rest of our server room.

I'm curious if this setup conformed to Fig. 5 in RFC 5598.
MDA  --> Mailstore --> Recipient

I suspect this is a case where the Mailstore would be "off to the side" in
a Relay-level diagram, and connected via a secure, high-bandwidth line
crossing the international border.
MDA --> Recipient
 //
Mailstore
If you slavishly follow the RFC, then the Mailstore has to replicate all
the functions of the Message Delivery Agent, authenticating Recipients,
running POP, IMAP, and HTTP, etc., and will likely be a lot less secure.

---

From: David MacQuigg <[email-masked]>
Date: Sat, 28 Sep 2019 02:53:51 -0700

Hi John.  It's been a few years since we met at UofA.  Good to see you are
still active with email.  Thanks for your thoughtful response below.  I
would love to see a three-lecture series like you describe, and I would
gladly drop some other material to make room.  Unfortunately, as a mere
"Industrial Associate" I was never able to squeeze in more than one lecture.

You make a good point about the complexity of real-world systems, and the
fact that our classroom presentations can't really capture all the
important details.  There is always a compromise between being informative
and keeping it simple.  http://en.citizendium.org/wiki/Email_system is my
best shot at a one-lecture compromise.  Figure 4 in RFC 5598 is simple, but
not very informative.  Fig. 5 is informative, but not simple.

Most of what I see in computer network textbooks errs on the side of messy
detail.  What's important at this level is focus on fundamentals and
building a solid foundation of understanding.  I don't let students think
that my presentation is the whole story.  I show them Fig. 5 in RFC 5598 to
let them know there is lot more to learn if they ever get serious about
email systems.  I firmly believe that students are better prepared to learn
these details, if they start with a good understanding of the fundamentals.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 28 Sep 2019 15:35:25 +0530

> was authored by a person.  That email was sent through a web page.

Thanks for clarifying.

Ok.

>
> According to your proposal (Page 240),

I have used C/R only as an example. C/R part is optional. I have provided
other methods for C/R alternatives. e.g. Proof-of-Work.

Will all emails which are not DKIM or SPF "authenticated" be discarded?

"Chapter 6: Layers" section talks about 5 layers of my system. Encryption
layer (TLS), Authorization Layer (SPF), Alias Layer (SAD), Authentication
Layer (DKIM) and Alignment Layer (DMARC). When I originally designed the
system I mandated all 5 layers for certain type of boxes. So all 5 layers
result must be "Pass" to accept mails. But later I relaxed the requirements
only to Authorization Layer (SPF) and Alias Layer (SAD). You can see that
in Page 298 under "amendments" section. So DKIM is optional in my system.

For validation, we primarily rely on Authorization Layer (SPF). For
Domboxes, we check only the SPF record. But for Restricted Mailboxes we
give preference to the MX record. We first extract the MX record IP
addresses and compare with the Client IP. Then we go for SPF record, A
record and third-party MX host SPF record.  The "Authorization Layer"
result is set during the "MAIL FROM" command. It can be one of: PASS,
NEUTRAL, FAIL

The mail rejection happens during the RCPT TO command. We first decide the
RCPT TO address box type. If it is a Restricted Mailbox, then it requires
"Pass" result for "Authorization Layer" to proceed. If "Authorization
Layer" result is "Pass", then everything looks fine so far. If not, then we
check whether "MAIL FROM" address is found in the recipient address book.
If found, then we fast-track the mail even though "Authorization Layer"
result is not "Pass".

If neither verified nor authorized in the address book, that's when we
reject the mail with an error message like "550 Restricted Box.
Unauthorized and Unverified Sender. Please configure SPF or Send this mail
from one of your MX server IP address"

To answer your question, Challenge mails will be triggered only on the
following conditions. (a) The sender is a verified stranger (b) The sender
is not found in the recipient address book.  If those conditions are not
met, then the mail will be rejected with said error message.

Most botnet mails are from forged "from" addresses. So our "Authorization
Layer" result would be either "NEUTAL" or "FAIL" for botnet mails. If the
result is "PASS", that's when we go for "domain registration date" to rate
limit mails from fresh domains.

Hope that helps.

-- 
Best Regards,

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Sat, 28 Sep 2019 13:49:43 -0400

> other methods for C/R alternatives. e.g. Proof-of-Work.

Proof-of-work is unusable in a world where the Bad Guys(TM) already run
botnets doing bitcoin mining - and people even willingly visit websites that
are quite open about using the CPU of the visitor to do mining while the visitor
is looking around the site.

That's the problem with proof-of-work - it only proves that *somebody* did
the work.

Also, if you do the math, it turns out that using a proof-of-work that's challenging
enough to dissuade Bad Guys(TM) is also so high as to totally cripple outbound
mail servers for many legitimate uses.

For example, vger.kernel.org hosts a number of Linux kernel related mailing
lists. The main linux-kernel list alone has 5,000 subscribers and 1,000 posts per
day.  Even if you ask for only 1 CPU second as a proof-of-work, that's keeping
60 cores busy 24/7 doing nothing but proof-of-work for one mailing list.

And one CPU second is trivial to owners of a botnet, even if the botnet isn't what's
doing the actual mailing.

> from one of your MX server IP address"

Again, if you're trying to prevent spam, this doesn't work in a world where the
spammers often have more squeaky-clean SPF and DKIM and DMARC than actual
production mail servers.

You're confusing "authentication" with "authorization" - the fact that you've
"identified" the "source" of the mail doesn't tell you if you should actually accept
mail from that address. And strictly speaking, this is pretty weak "identification",
as it relies entirely on credentials created by the entity trying to convince you of
their identity.

(Wait till you start digging in to fast-flux A records, and even worse, fast-flux NS
records for DNS - the Bad Guys have been known to flip DNS records every few
seconds, so half the time an A record points at something that has a matching PTR
entry that looks legitimate, and the other half the time it points at a pwned machine
that's part of the botnet.  And they'll even do this with their NS records, just to make
it hard to figure out what's going on.)

Before you ask:  Yes, they've been seen to fast-flux SPF records as well.

And has has been pointed out *several* times, "send from MX" is often unworkable
as well.

---

From: "John Levine" <[email-masked]>
Date: 28 Sep 2019 20:00:50 -0600

>is trying to swindle me (latest iphone 3€!!!).

Comparing the amount of spam I get from Sendgrid and from Mailchimp,
I'm reasonably sure that if the former put as much effort into
managing its outgoing mail stream as the latter does, we'd see equally
little spam from them.  There are similar differences among URL
shorteners and hosting providers which lead me to similar conclusions.
(My VPS are hosted at a place called Tektonic which you've probably
never heard of because their outgoing spam filters work very well.)

Spam reporting actually works pretty well, but by the time a largish
sender gets a report back, the spam run is probably over.  Reactive
approaches are fine for checking how well your tools work and tuning
them but it's not an effective way to keep the stuff from going out.

R's,
John

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sun, 29 Sep 2019 11:27:15 +0530

> That's the problem with proof-of-work - it only proves that *somebody* did

Proof-of-Work is one of our injection methods. All the injection methods
(e.g. CAPTCHA, Proof-of-Work etc.) in our system is applicable only to
"verified strangers". So this *somebody* somehow tied to a domain. Abusing
the system means, domain reputation takes a hit.

And yes bad guys already mining bitcoin via botnet. If you tell them, they
have to give away some CPU, then they are not gonna worry about that much.
But if you tell them, they have to give away 1 cent worth of bitcoin that's
when they start to see the value.

> doing the actual mailing.

You are talking about bulk mailing here. Our "injection" phase is not
designed for bulk mails. That's why we throw this warning when the user
enabled restricted mode.

> (b) No, Get me out of here.

So the problem here is not with the system I designed. But with the "user
education". Every new system requires some kind of learning curve. So you
can't blame me for that.

I certainly don't want the end user to enable restricted mode and give out
their email address to ietf.org mailing list. I want them to create an
isolated mail address which would look like [email-masked]
and then give this address while signing up to ietf mailing list. That
address gives exclusive mailing privilege to ietf.org. No captcha, No
proof-of-work etc.

>
> You're confusing "authentication" with "authorization"

I coined terminologies differently for my system.

Authorization Layer (SPF) - Checks whether the “Sending IP / Client IP” is
*authorized* to send mails for the “Envelope
Domain”.
Authentication Layer (DKIM) - This layer checks whether the mail is
digitally signed
Alignment Layer (DMARC) - Checks whether the domains are aligned

I even created a story to explain these layers to end users.
https://medium.com/@Viruthagiri/the-kings-story-6df87b32b80f

(Wait till you start digging in to fast-flux A records, and even worse,
> fast-flux NS

 I don't have any experience with fast-flux. I have to educate myself on
this. Thanks for guiding me.

-- 
Best Regards,

---

From: David MacQuigg <[email-masked]>
Date: Sun, 29 Sep 2019 05:56:00 -0700

Changing the subject to minimize discussion of technical detail.  I would
just like to hear from others with more experience - which of the existing
services are doing it right, and how do they do it?  The goal is near-zero
spam in my inbox (1% is my threshold of annoyance), and no lost emails.
Also, please disclose if you have any connection with the service.

I have been using gmail for the last few years.  No connection other than
as a client.  Spam in my inbox is maybe one in 200.  I have noticed one or
two lost emails in the last year.  Before that I was not aware of any.
Note:  I don't count as spam any that appear on a separate tab labeled
"Promotions".  I don't count as "lost" any that appear in my spam bucket.
How do they do it?  I guess they have a very complete whitelist of
senders.  They also have an icon "Report spam" at they top of their
webpage, which I assume is their main source of rapid feedback.

My wife uses Yahoo, and she is seeing a lot more spam, but not enough to
motivate her to change services.  I guess they also use a whitelist.

---

From: Russ Allbery <[email-masked]>
Date: Sun, 29 Sep 2019 08:19:22 -0700

David MacQuigg <[email-masked]> writes:

> I have been using gmail for the last few years.  No connection other
> than as a client.  Spam in my inbox is maybe one in 200.  I have noticed
> one or two lost emails in the last year.  Before that I was not aware of
> any.  Note:  I don't count as spam any that appear on a separate tab
> labeled "Promotions".  I don't count as "lost" any that appear in my
> spam bucket.  How do they do it?

Gmail sees something like 20-30% of all email on the Internet and has
millions of users manually classifying false positives and false negatives
for them in real time in their application.

There are obviously a whole bunch of other technical controls, resources
to build large databases of vendors, and so on that are hard to reproduce
economically, but I think the first point is the critical one.  Spam is a
classification problem, your classification engine gets better and better
the more data you have (assuming that you have the engineering resources
required to do something with that data, which Google does), and Gmail has
by far the best source of data of anyone doing email.

The things that many people dislike about Gmail (central concentration of
email clients and senders into one powerful entity, vast visibility into
everyone else's business, the ability to gather surveillance information
through network effects) are exactly what makes them so good at spam
filtering.

-- 
Russ Allbery ([email-masked])              <http://www.eyrie.org/~eagle/>

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sun, 29 Sep 2019 17:57:34 +0200

On Sunday 29 September 2019 14:56:00 CEST, David MacQuigg wrote:
> I have been using gmail for the last few years.  No connection 
> other than as a client.  Spam in my inbox is maybe one in 200.  
> I have noticed one or two lost emails in the last year.  Before 
> that I was not aware of any.  Note:  I don't count as spam any 
> that appear on a separate tab labeled "Promotions".  I don't 
> count as "lost" any that appear in my spam bucket.  How do they 
> do it?  I guess they have a very complete whitelist of senders.  
> They also have an icon "Report spam" at they top of their 

Gmail uses an omnivore ML, so no whitelist. Rather, each sender has a 
database entry, much more complex than just "white" or "black". So "your 
order has been shipped" from example.com may be treated differently from 
"here are the new seasons's styles, just for you!!!" from the same sender, 
because gmail has inferred from its users's clicks that the two classes of 
messages are different in spamminess.

The previous paragraph is about 90% hearsay.

Arnt

---

From: S Moonesamy <[email-masked]>
Date: Sun, 29 Sep 2019 12:02:07 -0700

Hi David,
At 05:56 AM 29-09-2019, David MacQuigg wrote:
>Changing the subject to minimize discussion of technical detail.  I 
>would just like to hear from others with more experience - which of 
>the existing services are doing it right, and how do they do 
>it?  The goal is near-zero spam in my inbox (1% is my threshold of 
>annoyance), and no lost emails.  Also, please disclose if you have 
>any connection with the service.
>
>than as a client.  Spam in my inbox is maybe one in 200.  I have 
>noticed one or two lost emails in the last year.  Before that I was 
>not aware of any.  Note:  I don't count as spam any that appear

The various mechanisms discussed on the thread, and some others 
mechanisms such as content filtering, allow you to get close to that 
1% threshold.  You would have to assess whether those two lost emails 
are significant.  The problem with lost emails is that they end up 
being the ones which you might consider as important if you are using 
email for work.  There are other mechanisms which can be added.  They 
come with some disadvantages such as data exfiltration.  You would be 
giving some other entity visibility into your business.

The considerations are different for a home user.  For example, data 
exfiltration might not be an issue.  Profiling might be an 
issue.  You can find out more about that if you are using gmail: 
<https://takeout.google.com/settings/takeout>https://takeout.google.com/

Regards,
S. Moonesamy

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Sun, 29 Sep 2019 18:17:41 -0400

> designed for bulk mails. That's why we throw this warning when the user

So how does your system classify this email you're reading right now?

Is it spam, or bulk mail, or something the user wanted?

> I want them to create an isolated mail address which would look like
> [email-masked] and then give this address while signing up to
> ietf mailing list.

That breaks one of the big advantages of having a fixed address - recognizability
by others.  The address I'm using in this email has been in use for at least 2 decades
now, and allows others to tell that the "me" posting on this list is the same "me" posting
to a Linux mailing list.

That's why '+' addressing was invented.

A few user experience issues:

1) Many users will *not* think to create a new address for each mailing list
before they subscribe.

1a) Sometimes, it's not directly under the user's control.  For example, I end
up using the same address for email from Scouting USA National, and from my
local troop (because the troop roster is fed from National's records..)

2) You need to Do The Right Thing if somebody you've never interacted with does
an off-list reply to a post to a list.

3) If you want your scheme to work in the real world, it needs to play nice
with browser form autofill.

So... is there actually an architecture or proposal in that 300 page document,
or is it just a cargo-cult collection of things that people have used in the
past, to varying levels of success?

And are there any production workload numbers showing that it (a) scales and
(b) actually gets anywhere closer to "Zero" spam than most providers are
already managing?

---

From: Brandon Long <[email-masked]>
Date: Sun, 29 Sep 2019 16:59:01 -0700

If you'd like to know how Gmail works, the first step is Brad's original
paper on that:

https://ai.google/research/pubs/pub45

The system has evolved and become more realtime, larger, maintains
reputations on many more features, algorithms tuned, and various ML added
to the mix for some edge cases, but the system would still be recognizable
to him today.

As for size, I'm sure that there are a bunch of benefits of scale, but I'd
like to think we had better antispam even when we were much smaller than
the competition.

If anything, we're now such a target that entire so operations focus
explicitly on matching us.  If spam seems like a solved problem, that's
only due to dedicated work.  I guess the flip side of the benefits of scale
is that the antispam folks needed scales very sublinearly with the amount
of users covered.

Brandon

---

From: Hector Santos <[email-masked]>
Date: Sun, 29 Sep 2019 22:55:23 -0400

On 9/29/2019 7:59 PM, Brandon Long wrote:

> As for size, I'm sure that there are a bunch of benefits of scale, but
> I'd like to think we had better antispam even when we were much
> smaller than the competition.

Thank you for saying this. From a dimensionless analysis design and 
engineering standpoint, the only difference is higher levels of data 
collection of pretty much the same data we have all seen at any scale. 
  More here, less there, since 2003, I have automatic daily statistics 
collected for various "anti-spam" methods:

http://www.winserver.com/public/spamstats.wct

It has been pretty consistent and persistent.  IMV, any level of 
operation who follow the same deterministic protocols like SPF, 
should, in theory, see the same basic percentages I have seem since 
MARID. I have strongly focused on deterministic filtering methods. 
IMV, heuristics are not good for reliable rejections. I fundamentally 
don't feel we (as mail transport systems) should be in the 
"context-analysis" filtering business.  Our principle responsibility 
has been to move mail and bounce it when required.

Non-deterministic heuristics protocols and methods are still too high 
on false positives. Its good for always accepting with classification 
but I also see many false positives.  Gmail appears to have a problem 
with confusing support list systems as "bulk mailers."

My point, Google has not solved the spam problem even with big data, 
and I believe you conveyed that point.

-- 
HLS

---

From: David MacQuigg <[email-masked]>
Date: Mon, 30 Sep 2019 17:29:44 -0700

> https://ai.google/research/pubs/pub45

Yes, that article was excellent, basically proving everything I had been
advocating on this list and others, and also confirming what I was seeing,
on a much smaller scale with box67.com.  I wish the article had come out
two years earlier.  That might have saved me a lot of time.

> like to think we had better antispam even when we were much smaller than

It looked that way to me - one reason I dropped my experiment and switched
to gmail.  (The other being that the post-doc who was working with me on
plans to process the entire mailflow for U of A (74,000 recipients) got a
job offer he couldn't refuse.)

As for size, I'm surprised that nobody gave examples of small services that
are doing well.  I'm still convinced that size is not necessary to have an
excellent whitelisting system.  box67.com was just a few friends and
family, and we were whitelisting 90% of our incoming non-spam, sending the
rest through SpamAssassin.  That's a 10X improvement over SpamAssassin
alone.

> is that the antispam folks needed scales very sublinearly with the amount

Yes, once you have the system adequately automated, the labor required to
maintain Registry records is dramatically less.

Comparing box67 to gmail (as of 13 years ago):

1) box67.com used the HELO name to ID the sender, not the Mail From name.
The HELO name should identify the Transmitter (the ADMD just before the
Receiver (the first ADMD in the Recipient's Network)).
http://en.citizendium.org/wiki/Email_system
2) Default authentication records were based on SPF, A, and MX records.  We
didn't include PTR, CSV, or DomainKeys.
3) Reputation feedback was based on reports from Receivers, not button
clicks from Recipients.  We didn't have our own complete email service,
including webmail, which would have allowed us to use buttons reporting
spam and non-spam.
4) We had a webtool <http://open-mail.org/webtool.html> that Transmitter
domains could use to easily provide correct authentication information to
the Registry <http://open-mail.org/RegistryRecords>, and override any
errors in our default records.  A service as large as Gmail would not need
this, because they can more easily put the burden of proper authentication
on the sender.
5) Our goal was an open-source system that would benefit not just one big
service provider, but any organization that could be an honest and
competent participant.
6) Our statistics on Domain Ratings <http://open-mail.org/DomainRatings> were
nowhere near as complete and accurate as Google's (Taylor's Fig.1 and 2),
but they did generally agree with Google.  There is wide gap between
"spammy" and "non-spammy" domains, with very few in the middle.

Our stats also proved that we could build a very effective whitelisting
system based on reputation of the domain in the HELO command.  Most
legitimate domains send no spam from their authorized Transmitters.  Would
the rest of them eventually get on our whitelist?  The requirements are not
difficult.  As Bradley Taylor said "just authenticate and behave yourself".