Subject: [Asrg] Adding a spam button to MUAs

From: "John R. Levine" <[email-masked]>
Date: 9 Dec 2009 00:35:03 -0500

Most web mail systems have a spam or junk button that lets a user report 
unwanted mail to his ISP.  The ISP does whatever it does, typically tune 
their spam filters, and perhaps send a feedback report if the message is 
from someone with whom they have an FBL agreement.

Lots of us don't use web mail.  We use POP or IMAP to pick up our mail, 
and SUMBIT to send it.  How would we add a spam button in our MUAs work?

An obvious approach would be to pack up the message in an ARF report and 
mail it somewhere.  I don't think that would work, because MUAs these days 
can handle multiple inbound and outbound accounts, with the various 
accounts only loosely connected.  I have users who pick up their mail 
here, but send via their ISP's mail server and vice versa.  If you were to 
send the report via SMTP you might well send it to someone who'd never 
seen the message before.

So the report needs to be tied to the inbound account.  For IMAP accounts, 
a simple approach is to have an IMAP spam folder, and move the message 
there.  AOL does this in their IMAP access, so I suppose that makes it a 
de-facto standard.  POP is harder, since there's nothing I can see that 
would obviously do the trick.  If you could assume that the message was 
still on the server, you could have a JUNK command that provided the UIDL 
of the message to report, but in typical POP setups, the messages are 
downloaded and deleted from the server before the user sees them.  The 
alternative is to add a command to upload the junk message, e.g.

JUNK
   +OK send the message
blah blah copy of downloaded message blah blah
.
   +OK junk reported

That's workable, although it's slow since it has to upload the entire 
message, and it may be hard for MUAs to implement since they often add 
annotations to the downloaded messages that would confuse the server if 
handed back.

Yet another possibility would be a command for the POP server that 
provides an address to which to the MUA can send an ARF report, keeping in 
mind that the report may take a roundabout route if the MUA is set up to 
use someone else's SUBMIT server.  The address would presumably be obscure 
and time limited, with the user's mailbox somehow encoded into it, so that 
the server can recognize the report when it arrives, and to limit the 
chances of random spam that happens to arrive at the reporting 
addresses being misinterpreted as a junk report.

Any bright ideas?  Is there a way to make this work with POP that isn't 
an utter kludge?

R's,
John

---

From: Andrew D Kirch <[email-masked]>
Date: Wed, 09 Dec 2009 01:00:44 -0500

John R. Levine wrote:
> Most web mail systems have a spam or junk button that lets a user
> report unwanted mail to his ISP.  The ISP does whatever it does,
> typically tune their spam filters, and perhaps send a feedback report
> if the message is from someone with whom they have an FBL agreement.
>
> Lots of us don't use web mail.  We use POP or IMAP to pick up our
> mail, and SUMBIT to send it.  How would we add a spam button in our
> MUAs work?
Spamassassin and other filters are using mailboxes for training
spam/non-spam.  An add on for Thunderbird which automates sending to
these at a click of a button or builds into Thunderbird's spam marking
system would make spam reporting to baysian filters faster.  If we're
looking towards LART'ing this should ALWAYS be done manually.  Automated
e-mail sent to people that don't want it is the problem, creating more
based on a broken research script isn't the solution.

Andrew

---

From: Steve Atkins <[email-masked]>
Date: Tue, 8 Dec 2009 22:00:50 -0800

On Dec 8, 2009, at 9:35 PM, John R. Levine wrote:

> Most web mail systems have a spam or junk button that lets a user report unwanted mail to his ISP.  The ISP does whatever it does, typically tune their spam filters, and perhaps send a feedback report if the message is from someone with whom they have an FBL agreement.
> 
> Lots of us don't use web mail.  We use POP or IMAP to pick up our mail, and SUMBIT to send it.  How would we add a spam button in our MUAs work?
> 
> An obvious approach would be to pack up the message in an ARF report and mail it somewhere.  I don't think that would work, because MUAs these days can handle multiple inbound and outbound accounts, with the various accounts only loosely connected.  I have users who pick up their mail here, but send via their ISP's mail server and vice versa.  If you were to send the report via SMTP you might well send it to someone who'd never seen the message before.
> 
> So the report needs to be tied to the inbound account.  For IMAP accounts, a simple approach is to have an IMAP spam folder, and move the message there.  AOL does this in their IMAP access, so I suppose that makes it a de-facto standard.

It's also already supported by some MUAs. Mail.app for one. Sorta.

>  POP is harder, since there's nothing I can see that would obviously do the trick.  If you could assume that the message was still on the server, you could have a JUNK command that provided the UIDL of the message to report, but in typical POP setups, the messages are downloaded and deleted from the server before the user sees them.  The alternative is to add a command to upload the junk message, e.g.
> 
> JUNK
>  +OK send the message
> That's workable, although it's slow since it has to upload the entire message, and it may be hard for MUAs to implement since they often add annotations to the downloaded messages that would confuse the server if handed back.
> 
> Yet another possibility would be a command for the POP server that provides an address to which to the MUA can send an ARF report, keeping in mind that the report may take a roundabout route if the MUA is set up to use someone else's SUBMIT server.  The address would presumably be obscure and time limited, with the user's mailbox somehow encoded into it, so that the server can recognize the report when it arrives, and to limit the chances of random spam that happens to arrive at the reporting addresses being misinterpreted as a junk report.
> 
> Any bright ideas?  Is there a way to make this work with POP that isn't an utter kludge?

The general purpose way would be for the receiving MX to embed a reporting address in a message header such that the MUA would then forward the message to when the user hit the button. The only functionality it requires from the MUA is the ability to send email, which almost all MUAs already support.

A small amount of cryptography can make that immune against most failure modes.

The only obvious flaw is that if the MX doesn't support whatever header is added it would be possible for the original sender to add that header to the mail they send, meaning that a sender can have recipients send spam reports from MX operators that don't support the protocol sent directly to them, rather than to the MX operator.

That's not actually a flaw, I don't believe. Rather it's a fairly significant advantage, in that it allows FBL consumers who are sending email to opt-in to an ad-hoc feedback loop from their recipients. Yet it also allows MX operators to override that by removing the header, or by replacing it with their own FBL reporting address instead - without the MUA needing any special knowledge, nor more than a few tens of lines of additional code to support.

It'd mostly be a slight variant on the List-Unsubscribe: header.

Cheers,
  Steve

---

From: Daniel Feenberg <[email-masked]>
Date: Wed, 9 Dec 2009 01:31:36 -0500 (EST)

On Wed, 9 Dec 2009, John R. Levine wrote:

> their spam filters, and perhaps send a feedback report if the message is from 
> someone with whom they have an FBL agreement.
>
...

Feedback is going to be controversial in this group.

The first thing I would like to see was for email clients to automatically 
include headers for all mail forwarded to "postmaster" or "abuse". I have 
never gotten a complaint about incoming spam mail that included headers 
and it is generally difficult to instruct users, given the multiplicity of 
clients.

Daniel Feenberg

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 09 Dec 2009 12:32:49 +0100

Steve Atkins wrote:
>> MUAs these days can handle multiple inbound and outbound accounts, with the various accounts only loosely connected.  I have users who pick up their mail here, but send via their ISP's mail server and vice versa.  If you were to send the report via SMTP you might well send it to someone who'd never seen the message before.
>> 

If a server fetches mail from remote IMAP or POP accounts, it has to 
annotate those messages in order to recognize them and route them to 
the corresponding abuse-box. This may require to introduce a new (not 
necessarily standard) header field.

> The general purpose way would be for the receiving MX to embed a reporting address in a message header such that the MUA would then forward the message to when the user hit the button. The only fun…

We may reuse the authserv-id's from Authentication-Results header 
fields that the user trusts. This method implies the MUA has validated 
them, e.g. against the Received fields, and is thus preferable than 
using, say, Delivered-To. It provides for sending a report to multiple 
places, unlike the IMAP folder method.

Once the MUA knows which domain(s) take the responsibility of 
delivering a given message, it would be advisable to lookup an address 
list on the DNS and only resort to prepending "abuse@" if no record is 
found. This additional level of indirection brings a twofold 
advantage: on the one hand the MX has a chance to use a different 
address, e.g. [email-masked]; on the other hand, a 
third party can monitor abuse reports even if example.com is also the 
sending domain. To wit

  _abuse IN TXT "[email-masked],[email-masked]"

Example.com should do similar lookups when it forwards the report to 
the sending domain, eliding duplicates. Certifier.example, as part of 
its activity, should check that its agreed-upon addresses are present 
in the records of its certified domains.

Alternatively, synthesizing addresses from VBR-Info header fields may 
result in a similar set of recipients. However, separate validation 
would be needed to avoid sending the report to untrusted senders.

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 09 Dec 2009 11:55:49 +0000

--On 9 December 2009 01:31:36 -0500 Daniel Feenberg <[email-masked]> 
wrote:

> The first thing I would like to see was for email clients to
> automatically include headers for all mail forwarded to "postmaster" or
> "abuse". I have never gotten a complaint about incoming spam mail that
> included headers and it is generally difficult to instruct users, given
> the multiplicity of clients.
>
> Daniel Feenberg

That would make my life so much easier. Except in the quite common case 
when someone reports an email to an intermediary who doesn't use the 
postmaster address, or reports it to me at my personal address.

I quite like the idea of having the server add a "Report-Abuse" header, 
which would usually contain an email or web address. The client could make 
a POST report through a web form to a database, or forward the email with 
fill headers.  With a suitable AppleScript, it's feasible that we could 
already do this with Mail.app - given that we distribute customised OSX 
images.

-- 
Ian Eiloart
IT Services, University of Sussex
01273-873148 x3148
For new support requests, see http://www.sussex.ac.uk/its/help/

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 09 Dec 2009 12:06:48 +0000

...
>>  For IMAP
>> accounts, a simple approach is to have an IMAP spam folder, and move the
>> message there.

Apple Mail already has a "Junk/Not Junk" ("In/désirable","No 
deseado/Conservar", etc) button. It uses it to feed it's internal content 
filter. When enabled, the action taken when spam is detected can be (a) 
mark the mail as spam (it appears brown in the list) or (b) file into a 
Junk mailbox (which can be any mailbox designated for the purpose), or (c) 
custom actions which include redirecting (eg to a spamcop address) or 
running an AppleScript.

On an IMAP server, it would probably be better to file the message into a 
designated junk mailbox, and have some process on the server take whatever 
action the service provider thought reasonable. The only new thing required 
here is a way of communicating either (i) to the user; which mailbox should 
be used as for junk, or (ii) to the server; which mailbox I'm actually 
using for junk.

The latter is probably the more principled approach, especially given 
language concerns; my server is at a University with a high proportion of 
international students! Perhaps some RFC5464 METADATA attached to the 
mailbox could be used. It would require a new entry at 
<http://www.iana.org/assignments/imap-metadata/imap-metadata.xhtml>

A more flexible approach would be the definition of a new standard IMAP 
flag: \Junk or similar. That would permit GMail style monolithic mailboxes, 
and even use of multiple mailboxes for junk (cf Twitter's "Block" and 
"Report" buttons).

So, the user might see a "junk" mailbox, but the client would need to flag 
every message that was filed in there. It might even be sensible to define 
a \Notjunk flag for training purposes, and you might want to do something 
different with user-designated junk than with automatically filed junk 
messages, so that might require an additional flag, say \Userjunk or 
\Autojunk in order to make the distinction. IANA don't seem to maintain a 
list of flags, so this might require a change to the RFC, or a new 
extension.

> Any bright ideas?  Is there a way to make this work with POP that isn't

No ideas here, I'm afraid. This is the sort of thing that stops us offering 
POP to users.

> R's,
> John
> _______________________________________________
> Asrg mailing list
> [email-masked]
> http://www.irtf.org/mailman/listinfo/asrg

---

From: Seth <[email-masked]>
Date: Wed, 9 Dec 2009 13:04:52 -0500 (EST)

> and SUMBIT to send it.  How would we add a spam button in our MUAs work?

The problem is much trickier than that.

If the mail is forwarded at the user's request (e.g. college alumni
forwarding-only addresses), then the server to report it to is the one
that received it from outside the user's forwarding chain.  That's the
one whose spam filter can be tuned to reject without worrying about
vanished messages or backscatter.

Seth

---

From: John Levine <[email-masked]>
Date: 9 Dec 2009 20:26:10 -0000

>Feedback is going to be controversial in this group.

I don't see why.

>include headers for all mail forwarded to "postmaster" or "abuse". 

Let's assume that any reports sent by SMTP are in ARF format which (when
implemented correctly) includes the entire message at issue.

R's,
John

---

From: John Levine <[email-masked]>
Date: 9 Dec 2009 20:28:07 -0000

>that received it from outside the user's forwarding chain.

The web mail systems haven't solved that problem, so I don't think we
need to, either.  Any forwarder of any size has FBLs set up, so it will
get reports about forwarded mail via the target system.  That works OK
for me.

Short of a mythical global reputation system, I don't see how one could
divert reports to forwarders without also diverting them to spammers who
pretend to be forwarders.

R's,
John

---

From: Skull <[email-masked]>
Date: Wed, 09 Dec 2009 21:32:06 +0100

> pretend to be forwarders.

+1

-- 
Professional System & Network Sarcazzer
http://bofhskull.wordpress.com/

---

From: Daniel Feenberg <[email-masked]>
Date: Wed, 9 Dec 2009 16:26:00 -0500 (EST)

> implemented correctly) includes the entire message at issue.

I was not talking about automatically generated reports, but simply 
forwarding of messages by users to postmaster, which includes complaints 
about spam, DSNs and other items that users need help with. Often help can 
be provided only if the headers are available, and they are never 
included. I then write back to the user asking for headers, and they 
either drop the matter, or resend the message, again without the headers. 
It is a pointless exercise all around, and one easily fixed by the MUA 
vendors. I don't think users who can't send headers are going to master 
the art of creating ARF format complaints. In any case, I am not the 
source of the abuse, merely someone who would like to help out.

Daniel Feenberg

---

From: "Andrew Richards" <[email-masked]>
Date: 9 Dec 2009 22:05:42 +0000

On Wednesday 09 December 2009 05:35:03 John R. Levine wrote:
> de-facto standard.

When setting up mail server IMAP & POP3 settings there could be 'Spam 
reporting' options available - for IMAP this would presumably be the folder 
where junk is placed to be reported; for POP3 this might just be a checkbox to 
say that a JUNK command is implemented as John suggests.

> POP is harder, since there's nothing I can see that
> downloaded and deleted from the server before the user sees them.

An MTA providing a junk/spam reporting function might choose to keep [a copy 
of] all messages for at least a few days so that UIDLs refer to messages 
available to the MTA. This has the advantages,
 - The MTA wouldn't have to remove any headers added by the MUA
 - No significant bandwidth is required to communicate the UIDL (contrast
   this to the MUA having to send a message back to the MTA)

Perhaps MTAs that wish to provide spam reporting functionality should be 
required to keep [a copy of] messages for a few days, probably separate from 
the users' mailbox/folders view so that users/MTAs can still DELEte messages 
etc.

If a JUNK command comes in after the MTA's copy of the message has been 
deleted the command can just be ignored - most JUNK commands can be expected 
to be received within a few days of the user downloading the corresponding 
messages; or better still the MTA can inform the MUA whether it's been able to 
successfully find the corresponding message and report it / unsubscribe it etc.

> The alternative is to add a command to upload the junk message, e.g.
>
> JUNK
>    +OK send the message

I don't like this idea because of the waste of bandwidth (users may not 
appreciate sending back a large message if their upload bandwidth is low 
compared to their downstream - e.g. ADSL)

> Any bright ideas?  Is there a way to make this work with POP that isn't

cheers,

Andrew.

---

From: John Levine <[email-masked]>
Date: 9 Dec 2009 22:24:55 -0000

>reporting' options available - for IMAP this would presumably be the
>folder where junk is placed to be reported; for POP3 this might just
>be a checkbox to say that a JUNK command is implemented as John
>suggests.

POP and IMAP both have capability commands which allow the MUA to ask
the server what options it supports, so if we go this route, the MUA
support could be entirely automatic with the junk button appearing on
any mailboxes whose servers support the needed features.

I'm still wondering wither an IMAP spam folder and a POP
something-or-other option have any chance of gaining traction.

>An MTA providing a junk/spam reporting function might choose to keep
>[a copy of] all messages for at least a few days so that UIDLs refer
>to messages available to the MTA.

It's a reasonable thought, although the POP spec says quite clearly
that once the message is deleted from the server, the UIDL is no
longer valid.  It'd be a significant change to POP.

R's,
John

---

From: Seth <[email-masked]>
Date: Wed, 9 Dec 2009 18:57:30 -0500 (EST)

> pretend to be forwarders.

Easy: as the end user, I _know_ which sites I _told_ to forward for
me.

Seth

---

From: Steve Atkins <[email-masked]>
Date: Wed, 9 Dec 2009 16:05:05 -0800

> Easy: as the end user, I _know_ which sites I _told_ to forward for

In general, no you don't. (You might not care about the general case,
but it's there.)

That's beside the point, though. What is the disadvantage of sending
reports to spammers? If there is a disadvantage, how is that different
to the current FBL state of the art?

Cheers,
  Steve

---

From: John Levine <[email-masked]>
Date: 10 Dec 2009 00:23:53 -0000

>> Easy: as the end user, I _know_ which sites I _told_ to forward for

Indeed, but explaning that to your spam button seems unlikely to be
something that a normal mail user could reasonably configure.  One
could propose some auto-configure thing, but now we're back to how to
tell who's lying when they offer auto-configure stuff.

>to the current FBL state of the art?

I took Seth's suggestion to be to report to the forwarder rather than
directly to the ISP.  Sending it to both would be fine, give or take
issues of malicious misconfigurations mailbombing innocents.

R's,
John

---

From: Seth <[email-masked]>
Date: Wed, 9 Dec 2009 21:39:13 -0500 (EST)

Steve Atkins <[email-masked]> wrote:

> reports to spammers?

1. It confirms the address.

2. It doesn't go to the site where it would do some good.

Seth

---

From: der Mouse <[email-masked]>
Date: Wed, 9 Dec 2009 21:47:12 -0500 (EST)

>> That's beside the point, though.  What is the disadvantage of
>> sending reports to spammers?

Is there any reason to think spammers still care about this?

I can't see any reason for them to, I can see reason for them not to,
and I haven't seen anything but very old data being repeated to support
the implicit claim that they still do.

Also, my own SMTP sink gives me some small reason to think they don't
bother, for what it's worth.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		[email-masked]
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

---

From: Steve Atkins <[email-masked]>
Date: Wed, 9 Dec 2009 19:13:34 -0800

> 1. It confirms the address.

Nope. [1]

> 2. It doesn't go to the site where it would do some good.

Nor does it go anywhere where it would do any harm.

This is only in the case where the MX operator (and the forwarder or mailing list manager or anyone else along the delivery path) is not providing any sort of FBL, nor gathering TiS data, at least not by using this approach, so nobody else will be doing anything with the data.

Cheers,
  Steve

[1] Der Mouse expands on this.

---

From: Steve Atkins <[email-masked]>
Date: Wed, 9 Dec 2009 19:37:39 -0800

> This is only in the case where the MX operator (and the forwarder or mailing list manager or anyone else along the delivery path) is not providing any sort of FBL, nor gathering TiS data, at least…

Uh, to expand on that. I'm seeing the model in this case is that the spammer pretends to be a forwarder, so a report that might otherwise go to a forwarder would go to the spammer instead.

If the mail delivery path is "spammer -> forwarder -> isp -> recipient" then the only useful places to report it are the ISP, the forwarder and (potentially) the spammer. If, instead, it's "spammer-pretending-to-be-a-forwarder -> isp -> recipient" then the only useful place to send it is the ISP, but sending it to the spammer as well has no real cost.

I might have missed a suggestion to not send it to the ISP, only to the apparent forwarder, when there was something that looked like a forwarder - if so, don't do that. :)

Cheers,
  Steve

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 00:32:49 -0500 (EST)

> tell who's lying when they offer auto-configure stuff.

It shouldn't be that hard.  I tell the server "I've configured
seth@otherdomain to forward to this account."  The server sends a
magic message to seth@otherdomain; when it returns, the server can
observe the forwarding path and various header stuff, and configure
itself.  The only user input is the answer to "What other email
addresses are yours that you've set to forward here?"

Seth

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 00:39:16 -0500 (EST)

Steve Atkins <[email-masked]> wrote:

> If the mail delivery path is "spammer -> forwarder -> isp ->
> recipient" then the only useful places to report it are the ISP, the
> forwarder and (potentially) the spammer. If, instead, it's
> "spammer-pretending-to-be-a-forwarder -> isp -> recipient" then the
> only useful place to send it is the ISP, but sending it to the
> spammer as well has no real cost.

Problem: if the ISP gets the report and doesn't know that the
forwarder is legitimate, it will damage the reputation of the
forwarder.

> I might have missed a suggestion to not send it to the ISP, only to
> the apparent forwarder, when there was something that looked like a
> forwarder - if so, don't do that. :)

Not "looked like" but "was verified by the user, by the server sending
a magic message to the user-supplied 'this is my other address' and
seeing it come back from the forwarder".

Seth

---

From: John Levine <[email-masked]>
Date: 10 Dec 2009 06:21:04 -0000

>seeing it come back from the forwarder".

This feels too abusable.  We know that spammers sign up for sham
accounts at freemail services and send themselves lots of spam to game
systems that use a complaint percentage.  They could easily set up a
lot of fake forwards, which feel like they'd be useful for some kind
of statistics gaming.

Also, in the real world, there are legitimate forwards to people who
have only the dimmest concept that mail is being forwarded to them.
For example, my church has a temporary Sunday school director with an
AOL account.  His email skills are rudimentary, so rather than trying
to set him up in the IMAP or web mail that everyone else uses, I
forwarded the director's role account, which is in a lot of address
books and mailing lists, to his AOL account.  Getting him to do a
magic message thing would be non-trivial.  I have several other people
in similar situations, role accounts forwarded to people without the
skills or interest to learn another mail system, so I don't think this
is an unrealistically exotic concern.

FBLs feeding back to forwarders are existing art.  They're not
fabulous, but they're adequate, and I'd really rather not get
sidetracked again by the forwarding issue.

R's,
John

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 01:33:43 -0500 (EST)

John Levine <[email-masked]> wrote:

> accounts at freemail services and send themselves lots of spam to
> game systems that use a complaint percentage.  They could easily set
> up a lot of fake forwards, which feel like they'd be useful for some
> kind of statistics gaming.

But the forward would only be validated for those users who validated
it, so the spammers gain only the ability to spam their own accounts.

> For example, my church has a temporary Sunday school director with
> an AOL account.  His email skills are rudimentary, so rather than
> trying to set him up in the IMAP or web mail that everyone else
> uses, I forwarded the director's role account, which is in a lot of
> address books and mailing lists, to his AOL account.  Getting him to
> do a magic message thing would be non-trivial.

Does he know the address of the role account?  That's all he'd need.

(Also, the system need not work for everybody; 10% information is a
lot higher than providers get now.)

Seth

---

From: John Levine <[email-masked]>
Date: 10 Dec 2009 07:17:30 -0000

>it, so the spammers gain only the ability to spam their own accounts.

If the ISP is counting messages, all those "forwards" with no complaints
could screw up the statistics.

>Does he know the address of the role account?  That's all he'd need.

I could him the address, but then I'd have to explain why I wanted him
to do this thing when he's already getting his mail, and then where to
find the setup page at his ISP, etc. etc.  Like I said, it's
non-trivial.

>lot higher than providers get now.)

It's marginal and it's a distraction.  Any suggestions on how to make
a working spam button for POP accounts?

R's,
John

---

From: Skull <[email-masked]>
Date: Thu, 10 Dec 2009 09:02:38 +0100

> forwarder is legitimate, it will damage the reputation of the

This is obvious, but it's up to the ISP preventing this. Once they have
the message with full headers, he's expected to see what's inside it and
which path it took to reach his network, not only look at the last hop
and fire...

Expecting to find a techincal solution to let *end users* correct their
ISPs misbehaviour doesn't seem a good path to me...

---

From: Skull <[email-masked]>
Date: Thu, 10 Dec 2009 09:05:32 +0100

> addresses are yours that you've set to forward here?"

You're missing role-address aliases with several recipients. And mailing
lists like this!

I really hope people doesn't start sending similar probe messages to MLs
with thousands of recipients... :-P

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 10 Dec 2009 10:23:06 +0000

--On 9 December 2009 20:26:10 +0000 John Levine <[email-masked]> wrote:

>> "abuse".

I think that was a suggestion orthoganol to the issue at discussion. It's 
not about use of a "spam" button, but about any message forwarded to 
"postmaster", for any purpose.

But, ARF formatted reports would achieve this when the spam button is hit.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 10 Dec 2009 10:28:55 +0000

> forwarder is legitimate, it will damage the reputation of the

Forwarders who aren't taking care about what they forward *should* have a 
damaged reputation.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 10 Dec 2009 10:31:33 +0000

--On 10 December 2009 06:21:04 +0000 John Levine <[email-masked]> wrote:

> magic message thing would be non-trivial.

Right, so the responsibility should be upon the person or system setting up 
the forwarding. Of course, it should require confirmation by the recipient.

---

From: Skull <[email-masked]>
Date: Thu, 10 Dec 2009 12:17:19 +0100

On 12/10/09 11:28 AM, Ian Eiloart wrote:

> Forwarders who aren't taking care about what they forward *should* have
> a damaged reputation.

There's also people, out there, asking loudly for filter-free accounts
and redirecting it on another (filtered) account, just to hit the spam
button for every junk they receive from the forwarding.

Or, similar story: company here in Italy with remote office in US. info@
role account is an alias sending to both Italian people and to an AOL
account used by the remote office.
Italian users give opt-in in order to their suppliers in order to
communications from their Italian suppliers. The US guy hits the spam
button for each one he receives. I (the forwarding ISP) receive AOL's
FBL for each... :-\

Who's reputation is expected to be damaged? ;-)

---

From: Skull <[email-masked]>
Date: Thu, 10 Dec 2009 12:31:40 +0100

On 12/10/09 12:17 PM, Skull wrote:

> communications from their Italian suppliers.

I'd better re-read before hitting "Send", sorry... :-(
-- 
Professional System & Network Sarcazzer
http://bofhskull.wordpress.com/

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 10 Dec 2009 11:49:10 +0000

> Who's reputation is expected to be damaged? ;-)

The one who's forwarding unwanted email. Ideally, you're using SRS to 
forward from a specific address, so the reputation can be assigned to that 
address rather than the IP address. Then, when AOL stops accepting the 
email, the company will change their subscription. Perhaps you should 
forward the reports to info@...

If the messages contain DKIM sigs, then address based repuation is already 
feasible, but SRS and SPF would make it feasible otherwise.

It's for examples like this that you need a "Block" as well as a "Report 
for spam" button - like Twitter has.

---

From: Skull <[email-masked]>
Date: Thu, 10 Dec 2009 14:25:47 +0100

On 12/10/09 12:49 PM, Ian Eiloart wrote:

> forward from a specific address, so the reputation can be assigned to
> that address rather than the IP address.

So, again, "doing the right thing" with spam button complaints is up to
the ISP, not the end-user.

No reason to implement a "forward management" inside "The Spam Button
Feature", then.

IMHO.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 10 Dec 2009 14:23:39 +0000

> the ISP, not the end-user.

Generally speaking, the user may know what kind of complaint they have. 
However, the service provider (I'd say ESP, not ISP), may be in a better 
position to know how to handle the complaint. When forwarding, there are 
two service providers.

Service providers and software developers have a responsibility to enable 
users to do the right thing (ultimately, something that will stop the email 
at source), and to present a comprehensible user interface. The user 
shouldn't have to understand how an email got to them.

> No reason to implement a "forward management" inside "The Spam Button

That depends on its purpose, and how its implemented.

---

From: Dan Oetting <[email-masked]>
Date: Thu, 10 Dec 2009 08:57:31 -0700

> longer valid.  It'd be a significant change to POP.

This is the same as with the IMAP server. If a message is deleted from  
the server it can no longer be reported to that server as junk. When  
the user configures junk reporting on the MUA, the user would get  
notice of this requirement and can configure the retention period  
which is also a function of the MUA.

MTA's may take a more active role in pre-filtering the users mail and  
flag messages as junk before the MUA downloads the message content.  
MUAs that take advantage of this capability would download only select  
headers of such flagged junk giving the user control to download the  
full message or letting the MUA delete the messages from the server  
after a configured time has elapsed.

-- Dan Oetting

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 13:08:14 -0500 (EST)

> could screw up the statistics.

Since the ISP knows that the user validated them as forwards, they
should have no effect on deliverability to anyone else.  That includes
(not) using them in statistics.

> a working spam button for POP accounts?

How much information does the server retain?  Perhaps it should retain
more for "suspected spam" messages, and use whatever it has to at
least affect the amount of info retained for similar messages in the
future.  (Or the client could upload some amount of data, whatever is
most useful.)

Seth

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 13:16:45 -0500 (EST)

> ISPs misbehaviour doesn't seem a good path to me...

Who else can do it?  Whatever headers a legitimate forwarder sends, a
spammer can fake exactly; the only difference is the connecting IP
address.

Seth

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 13:19:18 -0500 (EST)

>> magic message to seth@otherdomain;
. . .
> You're missing role-address aliases with several recipients. And mailing

I don't think a mailing list is "one of my other addresses".  A role
account with multiple recipients is a mailing list.

> with thousands of recipients... :-P

Why would they?  (And since the probe messages would have a known
format, ML servers could easily reject them.)

Seth

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 10 Dec 2009 19:25:42 +0100

>> that received it from outside the user's forwarding chain.

The concept of _server responsible for accepting a message_ is what we 
  should use here. There may be multiple ones. How to validate them, 
etcetera, may be defined afterwards (that's the stance of RFC 5451.)

> get reports about forwarded mail via the target system.  That works OK

Not all forwarding targets provide for a FBL. Web mail and FBLs are 
local arrangements that can hardly be standardized so that they scale 
well to any mail system. I don't see many European FBLs, for example...

> pretend to be forwarders.

I thought such _mythical global reputation system_ is what we are 
researching right now. IMHO there is a necessary relationship between 
abuse reports and MGRSs: V can meaningfully vouch for X only if V can 
monitor _all_ ARs originating from mail emitted by X. (see "A Vouch By 
Feedback proposal" 
http://www.ietf.org/mail-archive/web/asrg/current/msg15679.html)

Even a user-trusted authserv-id validated against Received lines can 
be used to divert ARs: for example Alice may configure X as a trusted 
server, but mail to Alice@Y with forged Authentication-Results and 
Received header fields may still fool her MUA. While this may not be a 
problem now, as Steve said, it feels safer to be able to avoid it, 
just in case it will.

---

From: Seth <[email-masked]>
Date: Thu, 10 Dec 2009 13:43:36 -0500 (EST)

> Forwarders who aren't taking care about what they forward *should* have a 

Suppose they _do_ take care, and act according to the user's expressed
wishes to have everything forwarded?  (This makes more sense when
there's a chain involved, I give out Address1 which forwards to
Address2 which forwards to Address3 where I read mail.  I want
Provider2 to forward everything; Provider1 should do spam filtering,
including that enhanced by the information I provide from Address3.)

Seth

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 10 Dec 2009 19:46:51 +0100

Dan Oetting wrote:
>> the POP spec says quite clearly 
> the server it can no longer be reported to that server as junk. When the 
> user configures junk reporting on the MUA, the user would get notice of 
> this requirement and can configure the retention period which is also a 
> function of the MUA.

IMAP requires no special arrangements, since it may use BURL (RFC 
4468, and Lemonade enhancements) if bandwidth is a problem. A message 
may be moved to a junk folder before reporting it, for limited 
retention issues.

For POP3, I see no way but resending the whole message.

---

From: John Levine <[email-masked]>
Date: 11 Dec 2009 01:18:55 -0000

>researching right now.

No, but if you'd like to set up a reputation system subgroup, that
would be fine.

R's,
John

---

From: Douglas Otis <[email-masked]>
Date: Thu, 10 Dec 2009 21:53:28 -0800

> No, but if you'd like to set up a reputation system subgroup, that

Rather than a reputation system subgroup, something more along the line 
of a legitimate senders clearing-house system which establishes 
postmaster contacts, and tracks repeated feedback from vetted sources. 
It also seems that tracking and reporting a few other metrics not 
related to what is and is not spam, but rather levels of feedback, 
levels of repeated feedback, purported traffic volumes, and purported 
message categories, associated hostnames and signatures, would also be 
useful.

When moving to IPv6, only a system based upon who is legitimate can 
scale.  This approach might help mitigate claims of bias.  If this 
flies, then a labeling system could go with it. :^)

-Doug

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 11 Dec 2009 10:04:54 +0100

> No, but if you'd like to set up a reputation system subgroup, that

You mean setting up an ASRG subgroup? Of course I'd like to, it'd 
be... well, mythical: the missing link between authentication and 
reliability. It would be terrific to contribute that.

Would a new group boost discussions and possibly result in some 
practical outcome? I'd count Doug's refinement as a +1.

---

From: Ian Eiloart <[email-masked]>
Date: Fri, 11 Dec 2009 10:32:20 +0000

--On 10 December 2009 13:43:36 -0500 Seth <[email-masked]> wrote:

> wishes to have everything forwarded?

Then the two service providers need to have some kind of infrastructure 
that they can use to arrange for this to happen properly, mediated by the 
user of course.

So, when the user makes the request, the forwarder needs to be able to say 
either "I can't do that, it will damage my reputation when spam is 
forwarded", or "OK, but first you need to confirm the arrangement with the 
receiving service provider, by clicking this button".

What's required is some kind of forwarding protocol, perhaps.

> (This makes more sense when

---

From: John Leslie <[email-masked]>
Date: Fri, 11 Dec 2009 09:41:49 -0500

Douglas Otis <[email-masked]> wrote:
>> ... if you'd like to set up a reputation system subgroup, that
>> would be fine.
> 

   Neither John nor Doug should be surprised I'm still interested in
reputation systems.

   I remain convinced that senders need an established relationship
with vouching services and receivers need an established relationship
with reputation services, and that the interaction between these two
types of services is an area for interesting work.

   Both services individually, IMHO, could be mostly automatic, with
the reputation services receiving spam reports (presumably ARF format)
and notifying the appropriate vouching service when these pass a
threshhold set by the reputation service. (Passing the actual ARF
reports to the vouching services would not necessarily be allowed.)

   We would need to formalize how reputation services discover the
vouching service(s) related to specific senders, and it would help
to formalize how to report a complaint threshhold being passed. In
some cases, there should be sufficient trust between a reputation
service and a vouching service to pass each individual ARF report,
but IMHO this would not be the default case, and would need to be
supported by reporting an action taken.

   I'm not sure about the need to formalize how vouching services
discover the reputation services for particular receivers: I can
imagine cases where receivers would not want that information public,
and it's not obvious how this information would help.

   I do not believe it would be helpful to formalize how vouching
services report abuse allegations to their customers -- postmaster
and abuse emails are already quite useless for the majority of
domains, and IMHO anything we might standardize _will_ become
useless.

   Though I really don't want to formalize a reporting process, I
believe we should think in terms of a SPAM button automatically
sending an ARF report to the configured reputation service for that
receiver, which would discover the appropriate vouching service
and determine a reporting threshhold. When the threshhold is reached,
the reputation service would obfuscate anything it promised its
customers it won't reveal (and perhaps more if it has reason to
mistrust the vouching service), and pass the revised ARF report to
the vouching service. The vouching service would report to its
customer whatever it received (possibly reformatted for "easier"
interpretation by clue-challenged MTA operators) and possibly change
its vouching report to show "incident in progress." We presume that
if the sender proved completely unresponsive, the vouching service
would report "no longer our customer."

   YMMV, of course. Some large senders might want to operate the
vouching service internally, and some large receivers might want to
operate the reputation service internally; but this would tend to
complicate the N by M problem of determining the "reputation" of
vouching services -- that tussle, IMHO, will tend to work itself
out as vouching services fail to attain enough good-reputation
ratings.

--
John Leslie <[email-masked]>

---

From: Douglas Otis <[email-masked]>
Date: Fri, 11 Dec 2009 08:54:00 -0800

On 12/11/09 6:41 AM, John Leslie wrote:
> Douglas Otis<[email-masked]>  wrote:

The focus could be more on vetting feedback sources directed to the 
postmasters using _blind_ addresses, rather than assessing each 
individual message, and have a centralized feedback system that 
publishes related metrics and sender's specific information, such as 
their volumes, their purported types of messages, and their directly 
verifiable sources such as hostnames or DKIM signatures.  The direct 
information assists in establishing correctly attributed feedback.

The blind feedback address would need to verified with a ping-back from 
both the feedback address and that of the postmaster address, as a means 
to enable the feedback relay.  The system should also allow them to 
exclude sources of feedback sent to them and the level of consolidation, 
but this would not directly affect the metrics reported by the system. 
Such as repeated accounts, the number of unique signed feedback sources, 
reported volumes, and the number of message related feedbacks.

When a hostname is being relied upon for attribution, the system should 
report the addresses being used by that hostname with an upper limit of 
something like eight addresses.  Something as broad and nebulous as 
address lists can be heavily abused.  Those depending upon feedback 
vetted by hostnames need to ensure the hostname is within the indicated 
consistently used IP address range.  The postmaster should be able to 
directly publish this information, or allow the data to be gleaned, as 
not all sources will have established a contact.

> supported by reporting an action taken.

The relay system acting as a mindless intermediary could act as a type 
of assessment system, where metrics offer actionable information.  To 
keep metrics from being muddled, sources being heavily blocked would be 
excluded from published metrics.  There would also be a problem of 
detecting false "volume" assertions made by the sender, where this might 
be flagged by the number of unique feedback sources.  This suggests 
there might be a need for a daily volume report being feed back to the 
sender when more than N messages are received.  This might alert them to 
signature replay or hostname abuse.

> and it's not obvious how this information would help.

Let the feedback speak for itself, and have a system that acts as a 
conduit for feedback offer actionable information. Publicly announcing 
feedback email-addresses is normally heavily abused.  This system needs 
to ensure feedback remains relatively clean and actionable, but it 
should not find itself making decisions about any specific message being 
spam or not.  Each receiver using the metrics being collected can then 
apply their own thresholds.  By allowing largely legitimate senders the 
ability to block feedback sources, excluding bad sources can be 
automatic, without any direct examination of messages.  If anything, 
there could be a promise to never examine any particular feedback 
message, with the exception of volume reporting.

-Doug

---

From: John Levine <[email-masked]>
Date: 11 Dec 2009 23:19:17 -0000

>practical outcome? I'd count Doug's refinement as a +1.

If at least three people say they're interested, I'll set up a list
for it.

R's,
John

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 11 Dec 2009 16:27:09 -0700

> If at least three people say they're interested, I'll set up a list

I'm very interested in reputation systems (we're thinking about
deploying that on the XMPP network), but I'm not sure if we need a sublist.

Peter

-- 
Peter Saint-Andre
https://stpeter.im/

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 12 Dec 2009 10:59:14 +0100

Let me try and summarize the topics we are concerned about.

* "Assess operator's policy enforcement." IMHO, the primary product 
of feedback processing is to tell how cleverly an operator blocks 
reported spammers. However, an operator could lie, which is why we 
need the other topics.

>> types of services is an area for interesting work.

* "Relationship between vouching and reputation services." Vouching 
must clearly be based on reputation. Why should it be a separate 
service, and what are the relationships between them and with mail 
operators?

> information assists in establishing correctly attributed feedback.

* "Reliable feedback paths." This includes vetting sources, 
assessing operators, and establishing minimal authentication 
requirements for exchanging abuse reports.

* "Senders' generic profiling." A rather slippery slope, because we 
don't want to break anonymity nor privacy. But we need some metrics 
to identify "sockpuppetry" (or whatever we'd want to call multiple 
5322.From operated by a single individual/entity.)

Thus far, I've counted four topics. Did I forget or overgeneralize any?

If we want an ASRG subgroup, it is useful to pinpoint what we would 
use it for. To know where we are and what can we shoot for is 
obviously interesting anyway.

---

From: John Leslie <[email-masked]>
Date: Sat, 12 Dec 2009 10:31:30 -0500

Alessandro Vesely <[email-masked]> wrote:
>   reported spammers.

   It's not that simple, IMHO.

   Firstly, "reporting" of spammers is badly broken. If the only thing
you do is react to spam reports, your reputation is likely to be damaged
before the first report arrives.

   Secondly, the botnet problem is endemic. It's nearly impossible to
avoid having some ISP customers (or even enterprise users) let their
computers be infected and become part of a botnet. Blocking outgoing
port 25 helps, but it's very hard to catch botnet spam going through
your "standard" MTA before hundreds have gone out; and some botnet
operators are skilled at staying below your radar.

   Thirdly, fixing botnet-infected computers is insanely expensive to
ISPs _and_ enterprises, while botnet re-infection rates are scary.

> However, an operator could lie, which is why we need the other topics.

   I don't think we _can_ enumerate all the components of reputation
that matter. Certainly, tendency to mis-state policies and enforcement
matters a lot...

>   must clearly be based on reputation.

   Vouching based _solely_ on reputation is useless, IMHO.

   Vouching should be based on more immediate knowledge: are there
policies in place that satisfy the vouching service? What is the record
of enforcement of those policies? What measure do you have of "normal"
traffic for the sender you're vouching for?

>   Why should it be a separate service, and what are the relationships
>   between them and with mail operators?

   Vouching should be a separate service so that the relationship with
the (sending) customer isn't compromised. Different vouching services
should be able to vouch for different characteristics: percentage or
volume of emails considered abusive, responsiveness, etc...

>> information assists in establishing correctly attributed feedback.

   This is another item a vouching service might include: number or
percentage of emails received at spamtraps. Doug has a point here:
spam-reporting is _so_ broken that vouching services starting business
today probably would need a network of spamtraps to get useful reports.

   But IMHO spamtraps are not a viable long-term measure. It's too
easy to find out where a spamtrap is and ensnare an innocent MTA to
send something to it. (Some spamtraps can trigger on NDNs _required_
by the SMTP RFCs.)

>   requirements for exchanging abuse reports.

   Vetting spam-report sources is _hard_. Ideally, the reports would
contain enough information, and you'd have access to enough logged
information, to verify that such a message was indeed sent by the
customer of the vouching service.

   At first blush, that would seem sufficient to qualify the message
as spam, and your customer should agree to stop sending such messages
to that recipient. But of course you won't _get_ enough information
to say _what_ message and _what_ recipient without an established
trust level between reporter and vouching service...

   There's a delicate balance here, where a vouching service needs to
first satisfy its customer (the sender) and secondly assure the
reporter that it can be trusted with enough information to be useful.

>   don't want to break anonymity nor privacy.

   I don't believe this is a function for vouching services, but
rather a function for reputation services that don't have an
established relationship with the sender.

   IMHO the presence of vouching services reduces the need for sender
profiling.

---

From: Alessandro Vesely <[email-masked]>
Date: Tue, 15 Dec 2009 19:42:09 +0100

>    It's not that simple, IMHO.

Agreed :-)

> before the first report arrives.

This can be fixed by pushing, perhaps. Reporting huge quantities of 
spam may eventually move abuse-box operators.

> ISPs _and_ enterprises, while botnet re-infection rates are scary.

Botnet spam shouldn't be reported. It shouldn't be accepted in the 
first place, which would have brought us back to the question of how 
does one tell a zombie's disposable mail domain from a regular one, if 
we didn't know the answer: vouching. When whitelisting will work well, 
postmasters will be able to lower their spam thresholds without fear 
of FPs.

> that matter. Certainly, tendency to mis-state policies and enforcement

I've tried and depict it in 
http://wiki.asrg.sp.am/wiki/Abuse_Reporting . The reputation tracker 
appears there in all its invisibility ;-)
(There is also a tentative summary of the previous thread.)

>    Vouching based _solely_ on reputation is useless, IMHO.

Add the lapse of time that the mail domain has existed for, whether 
its whois information is complete, and similar "easy" data.

> traffic for the sender you're vouching for?

The policy that keeps jumping in my mind is "I'll ban you from sending 
for N minutes times the number of abuse reports I get about you."

I don't know about "normal" traffic. Perhaps, there could be an ARF 
field counting how many messages from a given IP a server got since 
the previous abuse report?

> volume of emails considered abusive, responsiveness, etc...

But how is that different from a reputation tracker?

> customer of the vouching service.

Yes. We won't catch professional spammers this way. We need to catch 
the good guys that occasionally fall into temptation. Numerically, it 
is a minimal part of spam, because most spam comes from botnet 
operators. Think of it as a sombrero-shaped pitch surface centered on 
ham: we want to block the neighborhood just outside the ham boundary, 
so that we can whitelist that region.

> trust level between reporter and vouching service...

When someone reports a message as spam, there are no concerns about 
confidentiality of the message's content. It may or may not be worth 
to avoid list washing capabilities by trying to protect the reporter's 
address; in such respect, it may have no sense to forward a report to 
both the sender _and_ its connection provider (steps 2 and 3 in the 
picture linked above.)

> reporter that it can be trusted with enough information to be useful.

IMHO, a voucher who determines that spammers are not being blocked 
should just withdraw vouching for the relevant mail domain --not doing 
so would degrade its own reputation. Maybe individual users deserve 
the right to appeal against a block, _they_ may want to know who 
reported what message as spam...

I agree a vouching service may cash a fee from the mail domain they 
vouch for. This does not imply corruption, though.

>    IMHO the presence of vouching services reduces the need for sender

An obvious workaround for a mailbox provider who has to block a user 
is to provide an alias. It should be forbidden. However, it is 
difficult to track it. Statistical data or sample audits may help to 
withdraw vouching for fraudulent operators.

Providers who don't even know who their users are, are unable to block 
them effectively. They are 2big2block, so vouching for them is useless 
anyway.

---

From: der Mouse <[email-masked]>
Date: Tue, 15 Dec 2009 14:22:10 -0500 (EST)

> Providers who don't even know who their users are, are unable to
> block them effectively.  They are 2big2block, so vouching for them is
> useless anyway.

As long as anyone is too big to block, we will have spamhauser.  Why?
Because spamming is lucrative, and those who are too big to block have
no incentive not to take advantage of that status.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: John Leslie <[email-masked]>
Date: Tue, 15 Dec 2009 14:46:41 -0500

> spam may eventually move abuse-box operators.

   I have no objection to you trying that...

   But I'd like to have a vouching service warn me _before_ I receive
any "abuse@" emails.

>> Secondly, the botnet problem is endemic...
> 
> Botnet spam shouldn't be reported...

   I don't agree. If one of my users is botted, I'd like to know it.
(Obviously, there are lazy operators who'd rather not know.)

> It shouldn't be accepted in the first place, which would have brought
> us back to the question of how does one tell a zombie's disposable
> mail domain from a regular one,

   I don't want _my_ MTA blacklisted because one of my users is botted:
I want to _seriously_ ration the damage he can do until he fixes it.

> if we didn't know the answer: vouching.

   But how does the receiving MTA know what vouching service to check?
And how does it know whether the vouching service is useful?

> When whitelisting will work well, postmasters will be able to lower
> their spam thresholds without fear of FPs.

   Agreed.

> http://wiki.asrg.sp.am/wiki/Abuse_Reporting
> 
> The reputation tracker appears there in all its invisibility ;-)

   A worthy exercise! I hope it gets updated as the discussion continues.

> its whois information is complete, and similar "easy" data.

   Be my guest... it still doesn't seem useful enough to me...

> for N minutes times the number of abuse reports I get about you."

   A useful policy, easily implemented... but what qualifies as an
"abuse report" sufficient to enforce this? (The majority of reports I
receive concern spam that never touched my network, for example.)

> the previous abuse report?

   Potentially useful... but you need the sender to agree to release
this information to your vouching service...

> But how is that different from a reputation tracker?

   Reputation trackers lack the sender-supplied information we have
discussed above.

> confidentiality of the message's content.

   We need reports from folks who can't be bothered to agree or disagree
with that.

> It may or may not be worth to avoid list washing capabilities by trying
> to protect the reporter's address;

   This is why I want to have spam reports go to a reputation service
which can satisfy (different) customen their privacy isn't compromised.
(I don't know what it will take to satisfy everyone...)

> in such respect, it may have no sense to forward a report to both the
> sender _and_ its connection provider (steps 2 and 3 in the picture
> linked above.)

   AFAICT, most spam-receivers figure it makes no sense to report to
_either_ of these. :^(

> so would degrade its own reputation.

   Agreed. There's a grey area in how long after a spam report is
forwarded before withdrawing vouching -- or how many spam reports before
withdrawing. This is why I envision a "vouching" report to say "incident
in progress," encouraging receivers to give a temporary error.

> Maybe individual users deserve the right to appeal against a block,
> _they_ may want to know who reported what message as spam...

   This is what we need to get away from: appeals are just _too_
expensive.

> vouch for. This does not imply corruption, though.

   There are actual expenses involved in vouching. A fee is needed to
cover these in most cases.

> them effectively. They are 2big2block, so vouching for them is useless 

   No provider is _always_ 2big2block! Some customers will want to avoid
blocking _some_ sending MTAs. That's a tussle for receiving providers
to work out with their customers. (And of course, nothing prevents a
reputation service from whitelisting "all" 2big2block senders...)

---

From: Douglas Otis <[email-masked]>
Date: Tue, 15 Dec 2009 16:14:49 -0800

On 12/15/09 11:46 AM, John Leslie wrote:
> Alessandro Vesely<[email-masked]>  wrote:
>> John Leslie wrote:
>>

John,

A problem that needs solving is defending specific feedback addresses. 
Bad actors will abuse email addresses intended for reporting abuse or 
contacting the postmaster.  While a vouching service can provide the 
desired feedback through indirect means, so can a feedback conduit service.

This service can:

a) Consolidate abuse feedback and limit the information exchanged.

b) Selectively allow blocking of feedback sources.

To obtain the service, users would allow their domain metrics to be 
published covering:

1) The amount of feedback obtained from "spam" reputation services.

2) The amount of account repeated feedback from "spam" reputation services.

3) The amount of feedback obtained from users "unwanted" email.

4) The amount of account repeated feedback from users "unwanted" email.

5) The amount of account and message volumes based upon message and 
volume specific feedback.

When more than 50% of senders blocked a source of "unwanted" email, it 
will not affect "unwanted" metrics.  It will also take 60 days before 
new "unwanted" email sources are included into the "unwanted" metrics.

[...]
>     Reputation trackers lack the sender-supplied information we have

This lost information could be replaced by having daily volume related 
feedback reporting.

>     We need reports from folks who can't be bothered to agree or disagree

Some protections can be achieved by using two categories of feedback. 
Feedback information primarily from spam-traps could be placed within 
the "spam" metrics.  Those whose feedback information is from user's use 
of "this is junk" buttons would be placed within the "unwanted" metrics.

> (I don't know what it will take to satisfy everyone...)

IMHO, a service that offers a conduit for feedback should:

1) only accept signed feedback reports.

2) ensure the postmaster of the domain authorized the feedback for their 
specific domain.

3) only publish general metrics, and never any specific message content.

4) assume those running spam services also use ghost systems for their 
reputation assessments.

5) assume only those running spam services offer feedback suitable for 
"spam" metrics, where all other metrics would be described as "unwanted".

[..]
> in progress," encouraging receivers to give a temporary error.

There would be fewer accusations of being unfair by only publishing 
metrics and allowing receivers to make their own assessments.  Some will 
indeed want a binary indication of whether to block email, but that 
falls into the bailiwick of anti-spam service providers, who will then 
need to defend assessments that indicate no email should be accepted. 
Of course the conduit metrics will also need to be defended against 
being gamed.  As such, metrics from any specific source could ensure 
volumes do not change by more than a small percentage over any given 
period.  Although this will be controversial, senders could share lists 
of sources issuing false reports.

>     This is what we need to get away from: appeals are just _too_

Publishing only metrics should avoid a need for handling appeals. Those 
being blocked would need to talk to the recipient blocking them.

> cover these in most cases.

Running just a feedback conduit service, information gained might help 
assist other email related services that could offset the related costs. 
  Consolidating reports also helps defray network expenses, by utilizing 
input more than output directions.

-Doug

---

From: Rich Kulawiec <[email-masked]>
Date: Tue, 15 Dec 2009 20:48:00 -0500

I think allowing end users access to such a button is a terrible idea.
This is the same population that routinely replies to spam, falls for
phishes, and fails to correctly execute rudimentary tasks like unsubscribing
from a mailing list or trimming quoted material from replies.  It would
be simpler and about as accurate to simply check a random number generator's
output for a "spam?/not-spam?" opinion.

---Rsk

---

From: Steve Atkins <[email-masked]>
Date: Tue, 15 Dec 2009 17:50:24 -0800

> output for a "spam?/not-spam?" opinion.

Data from actual reality contradicts your (otherwise plausible) reasoning.

Cheers,
  Steve

---

From: Douglas Otis <[email-masked]>
Date: Tue, 15 Dec 2009 18:32:01 -0800

> output for a "spam?/not-spam?" opinion.

If users are given any button, it should not say this is spam.  Instead 
it should say this is junk or unwanted.  When senders receive feedback 
from this category of sources, they might consider auto unsubscribing 
the messages that are from a mailing-list.  This assumes the source of 
the feedback is controlled by the postmaster, and that the account 
information is accurate.  The conduit does not need to know how the 
account specific information is confirmed.

The idea is to provide users a method where they do not need to 
unsubscribe themselves.  This can be dangerous.

-Doug

---

From: Franck Martin <[email-masked]>
Date: Wed, 16 Dec 2009 15:06:17 +1200 (FJT)

This is the story of the remote command for the VCR

After extensive studies with real people, and all the possibilities to record a channel while watching another channel and program recording at will. Manufacturers have realized people were using only one button: Record Now!

People don't want options, they just want simplicity.

Studies in email have also shown people use the spam button to just mean: I don't want to see these emails anymore.

The approach taken by google is interesting, because the report spam button, in the right conditions will act as an unsubscribe button, which can be easily supplant an FBL. I think this is what we are talking about here.

Franck Martin 
http://www.avonsys.com/ 
http://www.facebook.com/Avonsys 
twitter: FranckMartin Avonsys 

----- Original Message -----
From: "Rich Kulawiec" <[email-masked]>
To: "Anti-Spam Research Group - IRTF" <[email-masked]>
Sent: Wednesday, 16 December, 2009 2:48:00 PM GMT +12:00 New Zealand
Subject: Re: [Asrg] Adding a spam button to MUAs

---Rsk

---

From: Hrishikesh Barua <[email-masked]>
Date: Wed, 16 Dec 2009 09:26:11 +0530

>> In the right conditions will act as an unsubscribe button -

Correct me if I'm wrong - by the right conditions I understand that you mean
cases where the List-Unsubscribe header is present in the email. While this
is a good option, Gmail's implementation of this leaves some things to be
desired.

http://gmailblog.blogspot.com/2009/07/unsubscribing-made-easy.html

If you look at the screenshot in the post above, there is no just
"Unsubscribe" button - it's "Report Spam" or  "Unsubscribe and Report Spam".
In an update to the same post, Google clarifies that

>>If you want to unsubscribe without reporting the message as spam, click
"show details" in the top-right corner of the message, then click
"Unsubscribe from this sender."

If it's not among the visible options, people are not going to use it - they
are just going to click on the visible buttons (and mark it as spam).

But it's a start.

- Hrish
-- 
I dare to disturb the universe.
------------------------------------------
http://talonx.blogspot.com
http://code.deepinspace.net
------------------------------------------

On 16/12/2009, Franck Martin <[email-masked]> wrote:
> After extensive studies with real people, and all the possibilities to
> record a channel while watching another channel and program recording at
> will. Manufacturers have realized people were using only one button: Record
> Now!
>
> The approach taken by google is interesting, because the report spam
> button, in the right conditions will act as an unsubscribe button, which can
> be easily supplant an FBL. I think this is what we are talking about here.
>
> Franck Martin
> http://www.avonsys.com/

---

From: Ian Eiloart <[email-masked]>
Date: Wed, 16 Dec 2009 12:14:11 +0000

--On 16 December 2009 09:26:11 +0530 Hrishikesh Barua <[email-masked]> 
wrote:

> Correct me if I'm wrong - by the right conditions I understand that you
> mean cases where the List-Unsubscribe header is present in the email.
> While this is a good option, Gmail's implementation of this leaves some
> things to be desired.

IIRC, the message also has to have a valid DKIM signature, which covers the 
list-unsubscribe header. At least, there has to be some way that Google can 
tell that the message really is from the list.

> "Unsubscribe" button - it's "Report Spam" or  "Unsubscribe and Report
> Spam". In an update to the same post, Google clarifies that
>
> If it's not among the visible options, people are not going to use it -
> they are just going to click on the visible buttons (and mark it as spam).
>
> But it's a start.
>
> - Hrish

---

From: Rich Kulawiec <[email-masked]>
Date: Wed, 16 Dec 2009 07:07:42 -0500

> Data from actual reality contradicts your (otherwise plausible) reasoning.

Not my data.  I have a rather large collection of incidents involving
message recipients who have marked as spam:

	- personal messages
	- professional messages
	- system-generated messages
	- ordinary mailing list traffic
	- monthly mailing list you-are-subscribed reminders
	- mailing list subscription confirmations
	- etc.

In the case of the first several, this includes messages requested
by those same users or replies to their own.  In the case of the last
several, treating said false spam reports as "unsubscribes" has led
to indignant protests that they did not wish to be unsubscribed.
In the case of all of them, other-channel personal contact has almost
without exception led to denials that the button was used, followed by
admissions of error when proof demonstrating same was produced.

Users as a group are far too clueless and careless to be permitted the
privilege of access to such a feature.  If they weren't so clueless and
careless, then phishing would be an insignificant problem and outbound
mail logs would not show a constant stream of attempted replies and
unsubscribes to blindingly obvious spam.

But beyond that:

There's the zombie problem.  There is no way for anyone or anything
external to an end-user's system to know whether the button click (or
equivalent event) was generated by a user or by software working at
the behest of the new owner of the user's former system.  Given that the
zombie problem is epidemic and presently unstoppable, widescale deployment
of any such mechanism will lead to its use by zombie-resident malware as
soon as it's advantageous for abusers to do so. Thus, anyone proposing
such a "report as spam" mechanism on a large scale must also include in
their proposal a workable plan for solving the zombie problem.

---Rsk

---

From: John Levine <[email-masked]>
Date: 16 Dec 2009 14:55:33 -0000

>message recipients who have marked as spam:

Unless your collection is at least tens of millions of messages, I don't
think it counts as large.

More to the point, your collection has severe sample bias.  If you're
looking at incoming reports on a network that doesn't have bulk
senders and doesn't have a lot of consumer PCs that get botted, you're
not going to see many real complaints.  On my tiny network, 100% of
the feedback reports are either about COI lists to which the recipient
subscribed, or personal mail, since that's all it sends.  The reports
are still useful, both to prune out subscribers who aren't interested
any more, and these days, to identify spam sent from freemail accounts
that have gotten phished to lists to which the accounts are subscribed.

Large networks find user spam reports very useful, both on the reporting
network to tune their own sender reputation data, and on the receiving
networks to identify misbehaving senders.  This is reality, Steve and
I talk to people who manage the networks and this is what they tell us.

R's,
John

---

From: Paul Russell <[email-masked]>
Date: Wed, 16 Dec 2009 12:08:43 -0500

> Data from actual reality contradicts your (otherwise plausible) reasoning.

Our "actual reality" includes an AOL feedback loop stream that consists
primarily of:

* spam complaints about personal messages from family members or friends;

* spam complaints about replies to messages initially sent by the user who
  subsequently reported the reply as spam;

* spam complaints about messages posted to COI lists to which the user had
  previously and explicitly subscribed.

None of these messages should have been reported as spam, but they were, and
they outnumber legitimate complaints by a wide margin.

-- 
Paul Russell, Senior Systems Administrator
OIT Messaging Services Team
University of Notre Dame

---

From: John Levine <[email-masked]>
Date: 16 Dec 2009 18:42:01 -0000

>None of these messages should have been reported as spam, but they
>were, and they outnumber legitimate complaints by a wide margin.

You're in the same situation as Rich, a network that doesn't send
very much spam.  If you get so few reports that they're not useful,
none of the ISPs I know will have their feelings hurt if you turn
off the reports.

R's,
John

---

From: Seth <[email-masked]>
Date: Wed, 16 Dec 2009 13:59:04 -0500 (EST)

> message recipients who have marked as spam:

What fraction of the total spam complaints you received were bogus
like those?

> privilege of access to such a feature.

That depends on what the feature _does_.  A lot of inaccurate
measurements can be used to make a few very accurate measurements.

> external to an end-user's system to know whether the button click
> (or equivalent event) was generated by a user or by software working
> at the behest of the new owner of the user's former system.  Given
> that the zombie problem is epidemic and presently unstoppable,
> widescale deployment of any such mechanism will lead to its use by
> zombie-resident malware as soon as it's advantageous for abusers to
> do so. Thus, anyone proposing such a "report as spam" mechanism on a
> large scale must also include in their proposal a workable plan for
> solving the zombie problem.

How would it be advantageous for a zombie to report as spam?  Report
as non-spam, sure, to game the filters.  But with the data being noisy
to begin with, zombies adding noise don't have much effect; they might
require tuning of the filters.

Seth

---

From: Daniel Feenberg <[email-masked]>
Date: Wed, 16 Dec 2009 16:55:29 -0500 (EST)

> they outnumber legitimate complaints by a wide margin.

If you don't send any spam, then 100% of complaints will be bogus. The 
more revealing metric would be the ratios of proper and bogus complaints 
to the total number of messages sent. If you send a million messages (all 
non-spam) and get 4 bogus complaints, that is a failure rate of .0004%. If 
you take the ratio of bogus to true complaints, that is infinity (since 
there are no true complaints) or undefined. In any case, it isn't that 
interesting a number.

Your situation, where there are a handfull of valid complaints differs 
only in degree. The number of valid complaints does not belong in the 
denominator. The denominator should have the number of good messages. Then 
you can judge if the users are doing a good job or not.

Daniel Feenberg
NBER

---

From: Douglas Otis <[email-masked]>
Date: Wed, 16 Dec 2009 14:51:04 -0800

> require tuning of the filters.

Users without 0wned systems might still attempt to unsubscribe from 
spoofed subscriptions and be asked for passwords they never set, and 
then attempt to guess it anyway. There are also risks related to browser 
vulnerabilities that would be avoided by offering a "this is junk" 
button that invokes an unsubscribe service, even for user who have 
initially confirmed the subscription.

To avoid complaints, a web page associated with an email account could 
allow users a means to confirm their desire to unsubscribe, or just have 
user authentication included in the "this is junk" transaction, which 
might simply mean placement into the "junk" folder.  As such, it would 
be in the interest of list administrators to unsubscribe "unwanted" 
email based upon this feedback.

This feedback should not be confused with "spam" email feedback. 
Recently, new developers within our company confused these two 
categories and caused a number of complaints.  It is important to 
understand the difference between "unwanted" and "spam-trap" as 
determined by the source of the feedback.

Spammers will surely abuse any control mechanism in an effort to cause 
user complaints.  User complaints will cause the mechanism to be 
abandoned as being too expensive.  Users that are 0wned will likely be 
detected with spam-trap feedback, as well as through other mal activity.

Any effort to utilize email feedback MUST understand the difference 
between a general category of "unwanted" and feedback from "spam-traps" 
that are able to differentiate between "auto-responses" and DSNs.

-Doug

---

From: Nathaniel Borenstein <[email-masked]>
Date: Wed, 16 Dec 2009 18:20:56 -0500

I've tried hard to resist chiming in on this, but can't help myself.

I suggest that the issue of defining or identifying spam is a red herring and a huge distraction from actual progress.  The real issue is identifying mail that a specific user does not want.  When defined this way, it subsumes the spam problem, but it also maps more directly onto the user's action with a spam/junk button.  In this view, if a user clicks the spam/junk button for messages that we email gurus don't consider "spam," that's just fine -- each user is the expert in defining "junk" for their own mailbox.

As volumes continue to increase, I believe email streams need to be viewed more like intelligent news feeds -- we need to use sophisticated per-user information to choose which items to pass on to the user and which to block.  The spam/junk button can be very useful input to that process.  But there's more to it than that -- once we start doing our spam filtering with individual recipients in mind, we can do a lot of things right that we can't now.  For example, after twenty years of using spam filters, I still get spam in Chinese or Russian, languages I don't understand at all, because the spam filters know nothing about me as an individual.  Similarly, I'm sure that plenty of people in China get spam in English that seems obviously irrelevant to them.  This drives me nuts, because it would be so easy to fix if our filtering was done with even a minimal knowledge about the individual recipient.

In short, the real need is for user-focused email filtering to subsume our current notion of spam filtering.  Viewed that way, I think a spam/junk button would be an obvious win.   But it should be part of a broader interface (yes, an authenticated one, most likely) by which individuals could interact with their email filters, e.g. to say which languages they do and don't understand, or even which topics they were and weren't interested in receiving unsolicited messages about.  (Who knows, maybe there's even someone out there who doesn't want most spam but is desperate to enlarge the size of their private body parts; shouldn't they be able to communicate that fact to their filters?)  -- Nathaniel

On Dec 16, 2009, at 5:51 PM, Douglas Otis wrote:

> Users without 0wned systems might still attempt to unsubscribe from spoofed subscriptions and be asked for passwords they never set, and then attempt to guess it anyway. There are also risks related to browser vulnerabilities that would be avoided by offering a "this is junk" button that invokes an unsubscribe service, even for user who have initially confirmed the subscription.
> 
> To avoid complaints, a web page associated with an email account could allow users a means to confirm their desire to unsubscribe, or just have user authentication included in the "this is junk" transaction, which might simply mean placement into the "junk" folder.  As such, it would be in the interest of list administrators to unsubscribe "unwanted" email based upon this feedback.
> 
> This feedback should not be confused with "spam" email feedback. Recently, new developers within our company confused these two categories and caused a number of complaints.  It is important to understand the difference between "unwanted" and "spam-trap" as determined by the source of the feedback.
> 
> Spammers will surely abuse any control mechanism in an effort to cause user complaints.  User complaints will cause the mechanism to be abandoned as being too expensive.  Users that are 0wned will likely be detected with spam-trap feedback, as well as through other mal activity.
> 
> Any effort to utilize email feedback MUST understand the difference between a general category of "unwanted" and feedback from "spam-traps" that are able to differentiate between "auto-responses" and DSNs.
> 
> -Doug
> 
> 
> 
> 
>

---

From: Peter Saint-Andre <[email-masked]>
Date: Wed, 16 Dec 2009 16:33:25 -0700

On 12/16/09 4:20 PM, Nathaniel Borenstein wrote:

> each user is
> the expert in defining "junk" for their own mailbox.
> 
> As volumes continue to increase, I believe email streams need to be
> viewed more like intelligent news feeds

Interesting. Following the feed analogy, it's not even that "this is
junk" but "I don't want messages like this" or "show me more like this".

/psa

---

From: Michael Thomas <[email-masked]>
Date: Wed, 16 Dec 2009 16:00:41 -0800

I've thought much the same thing for years. To throw something else out, I think
that this is tied up with what I call "the alerting problem". That is, we are fed
many different streams of information these days (email, im, voice, news...) each of which
has their own filtering and prioritization -- if you're lucky. By alerting, I mean
the means that you become aware (or not) of a piece of communication vying for your
attention.

What I really want is to have the streams classified as is relevant to me, where
the alerting mechanism is tied to its *importance* to me, rather than the communication
channel. That is, there is nothing inherent with voice that should give it
preference to wake me from my sleep to hear an ad for the latest in garbage pail
liners. I want something to make sense of all of these things and raises things
to my conscious as makes sense for *me*.

Give me that, Nigerian scams and penis enhancement all pretty much take care
of themselves -- my classifier already knows that I could give a flying-f about those.

Mike

---

From: Douglas Otis <[email-masked]>
Date: Wed, 16 Dec 2009 17:05:45 -0800

On 12/16/09 4:00 PM, Michael Thomas wrote:
> I've thought much the same thing for years. To throw something else
> out, I think that this is tied up with what I call "the alerting
> problem". That is, we are fed many different streams of information
> these days (email, im, voice, news...) each of which has their own
> filtering and prioritization -- if you're lucky. By alerting, I mean
> the means that you become aware (or not) of a piece of communication
> vying for your attention.
>
> What I really want is to have the streams classified as is relevant
> to me, where the alerting mechanism is tied to its *importance* to
> me, rather than the communication channel. That is, there is nothing
> inherent with voice that should give it preference to wake me from
> my sleep to hear an ad for the latest in garbage pail liners. I want
> something to make sense of all of these things and raises things to
> my conscious as makes sense for *me*.
>
> Give me that, Nigerian scams and penis enhancement all pretty much
> take care of themselves -- my classifier already knows that I could
> give a flying-f about those.

Completely agree, when describing feedback originating from individual
users.  Your provider might be able to make sense of what is unwanted in
your view by observing messages marked by you as "junk".  They might
even extend this service by unsubscribing from mailing-lists, and
tracking repeat violations of expressed desires.

However, not all feedback would fall into the user category.  Feedback
obtained from spam-traps that recognize auto-responses and valid DSNs
offer better metrics when separated from user feedback of lesser quality.

The general concept was to offer a conduit service as a means to protect
the provider's feedback address from abuse, while at the same time,
allowing the publication of both user and spam-trap metrics as a means
to generally inform their recipients and management.

-Doug

---

From: Nathaniel Borenstein <[email-masked]>
Date: Wed, 16 Dec 2009 20:17:32 -0500

> offer better metrics when separated from user feedback of lesser quality.

I agree if you mean that each of these kinds of feedback is "better" when properly separated from the other.  I don't know that one is necessarily "better" than the other, but they convey different kinds of information and are therefore more useful when not mixed together.  I suspect that the user feedback will be most useful for high-level heuristic filtering, while automated feedback will permit certain kinds of more absolute filtering.  We need both (and more).

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 17 Dec 2009 16:26:37 +0000

--On 16 December 2009 12:08:43 -0500 Paul Russell <[email-masked]> wrote:

>>> phishes, and fails to correctly execute rudimentary tasks like
>>> unsubscribing from a mailing list or trimming quoted material from
>>> replies.  It would be simpler and about as accurate to simply check a
>>> random number generator's output for a "spam?/not-spam?" opinion.
>>
> None of these messages should have been reported as spam, but they were,
> and they outnumber legitimate complaints by a wide margin.

We get similar types of response, but the most common reports are about our 
Mailman mailing lists, and other bulk messages that we send. As far as I'm 
concerned they're legitimate complaints, but the volume would be much lower 
if (a) an "unsubscribe instead of complain" feature were implemented for 
lists, and (b) our alumni office would take more care with their lists 
(they're working on it).

I think the reason for a lot of these complaints is the completely hopeless 
AOL webmail interface for the feature. The "junk" button is right next to 
the delete button. Since the last time I looked (April '09), they've added 
about 5 pixels of separation between the two, but that separation hasn't 
always been there. I'd love to know what the mis-click rate is, and how it 
changed with the addition of some space. Subjectively, I've noticed a much 
lower rate of spam reporting from AOL, but can't say whether that coincided 
with the change, since I don't know when the change happened.

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 17 Dec 2009 16:27:25 +0000

--On 16 December 2009 18:20:56 -0500 Nathaniel Borenstein 
<[email-masked]> wrote:

> I suggest that the issue of defining or identifying spam is a red herring
> and a huge distraction from actual progress.  The real issue is
> identifying mail that a specific user does not want.  When defined this
> way, it subsumes the spam problem, but it also maps more directly onto
> the user's action with a spam/junk button.

Twitter seems to think that users are smart enough to distinguish between 
"unwanted" and "spam". They give you a button for each. It's an important 
distinction that most people can make. Even if they fail to do so, your 
information is at least as useful as a single button would give.

---

From: "Chris Lewis" <[email-masked]>
Date: Thu, 17 Dec 2009 12:06:21 -0500

John Levine wrote:
> not going to see many real complaints.

I explained that to RSK about a year ago.  Guess he forgot.

We've had a "this is spam" mechanism since 1997, averaging perhaps 
500-1000 hits/day over the past year (down from considerably more when 
our filters weren't as good).

It's quite accurate - probably well in excess of 99%.  We've never made 
an incorrect blocking decision based on an incorrect "this is spam" from 
one of our users.

On the other hand, we occasionally get one of our users accidentally 
marking an internal email as spam.  Those are unfailingly wrong.  Go 
figure ;-) I usually find out because I get a followon email saying "oops!".

It's a simple matter of sampling and biases.  If you emit a million 
emails per day, and 50% of it is spam, a 1% FP rate in spam reporting 
will probably not be detectable.  If you emit a million emails per day 
and _none_ of it is spam, a .00001% FP rate will make it look like all 
spam reporters are idiots worthy of derision on spam-l.

RSK falls into the latter scenario methinks (zero spamming, not "idiot" ;-)

---

From: Nathaniel Borenstein <[email-masked]>
Date: Thu, 17 Dec 2009 12:33:59 -0500

On Dec 17, 2009, at 11:27 AM, Ian Eiloart wrote:

> Twitter seems to think that users are smart enough to distinguish between "unwanted" and "spam". They give you a button for each. It's an important distinction that most people can make. 

Twitter isn't always right, and my intuition differs from yours on this one.  Fortunately it's something that could be resolved empirically.  I'd like to see such a study, because it wouldn't take very many users who *can't* properly make that distinction to render the two-button solution counterproductive.  I'd rather have one bit of meaningful data than two bits of muddled data.  -- Nathaniel

---

From: Ian Eiloart <[email-masked]>
Date: Thu, 17 Dec 2009 18:04:19 +0000

--On 17 December 2009 12:33:59 -0500 Nathaniel Borenstein 
<[email-masked]> wrote:

>> Twitter seems to think that users are smart enough to distinguish
>> between "unwanted" and "spam". They give you a button for each. It's an
>> important distinction that most people can make.
>
> Twitter isn't always right, and my intuition differs from yours on this
> one.  Fortunately it's something that could be resolved empirically.  I'd
> like to see such a study, because it wouldn't take very many users who
> *can't* properly make that distinction to render the two-button solution
> counterproductive.  I'd rather have one bit of meaningful data than two
> bits of muddled data.  -- Nathaniel
>

Well, then you have to make a judgment about what to do with the data. 
Application of fuzzy logic is essential here. You don't ban everyone who 
get's hit with a spam report, you wait til the reports hit a certain 
threshold, or you feed the message into your Bayesian analysis engine, or 
something.

What you can do, though, is stop delivering mail from the reported source 
to the recipient, perhaps after asking for confirmation. You need to be 
careful that this can be undone, though.

Yes, users make mistakes, and they have differing judgments about what is 
and is not spam, and those judgments may not be your judgments. But, that 
doesn't make the data useless, it just means that you have to think 
carefully about the application of the data in order to keep your false 
positive rates down.

Anyway, the worst case scenario is that users hit one of the two buttons at 
random. In that situation, you simply treat the buttons as equivalent, and 
take the same action for each.

---

From: Seth <[email-masked]>
Date: Thu, 17 Dec 2009 13:25:26 -0500 (EST)

>> between "unwanted" and "spam". They give you a button for
>> each. It's an important distinction that most people can make. 
>
> Twitter isn't always right, and my intuition differs from yours on
> this one.  Fortunately it's something that could be resolved
> empirically.  I'd like to see such a study, because it wouldn't take
> very many users who *can't* properly make that distinction to render
> the two-button solution counterproductive.  I'd rather have one bit
> of meaningful data than two bits of muddled data.  -- Nathaniel

One button is the "OR" of the two buttons, so there's no less
information available.  Given enough data, it should be easy to get
pretty accurate statistics on how reliable _each_ user is, and the
unreliable ones can be mapped into the one-button treatment.

Seth

---

From: Nathaniel Borenstein <[email-masked]>
Date: Thu, 17 Dec 2009 13:34:53 -0500

There's a difference between having an "A or B" bit, and having separate "A" and "B" bits that may be muddled.  If you don't know the right way to interpret the two bits, then the single bit is potentially much less misleading.

But as I said, this is something that could really be resolved by empirical study, and I'd prefer not to trust my intuition or anyone else's if we can figure out the facts about how people would actually use two buttons.  If the error rate is low enough, then two buttons will make sense, but I've rarely managed to underestimate the competence of the average user.   Two buttons is way more than twice as confusing as one, in this case.  -- Nathaniel

---

From: Paul Russell <[email-masked]>
Date: Thu, 17 Dec 2009 13:55:09 -0500

> take the same action for each.

If you're going to do that, why do you need two buttons?

---

From: Douglas Otis <[email-masked]>
Date: Thu, 17 Dec 2009 11:59:13 -0800

On 12/17/09 10:34 AM, Nathaniel Borenstein wrote:
> There's a difference between having an "A or B" bit, and having
> separate "A" and "B" bits that may be muddled.  If you don't know the
> right way to interpret the two bits, then the single bit is
> potentially much less misleading.
>
> But as I said, this is something that could really be resolved by
> empirical study, and I'd prefer not to trust my intuition or anyone
> else's if we can figure out the facts about how people would actually
> use two buttons.  If the error rate is low enough, then two buttons
> will make sense, but I've rarely managed to underestimate the
> competence of the average user.   Two buttons is way more than twice
> as confusing as one, in this case.  -- Nathaniel
>
>> Nathaniel Borenstein<[email-masked]>  wrote:
>>>> Twitter seems to think that users are smart enough to
>>>> distinguish between "unwanted" and "spam". They give you a
>>>> button for each. It's an important distinction that most people
>>>> can make.
>>>
>>> Twitter isn't always right, and my intuition differs from yours
>>> on this one.  Fortunately it's something that could be resolved
>>> empirically.  I'd like to see such a study, because it wouldn't
>>> take very many users who *can't* properly make that distinction
>>> to render the two-button solution counterproductive.  I'd rather
>>> have one bit of meaningful data than two bits of muddled data.
>>> -- Nathaniel
>>
>> information available.  Given enough data, it should be easy to
>> get pretty accurate statistics on how reliable _each_ user is, and
>> the unreliable ones can be mapped into the one-button treatment.

Agreed. After all, can users recognize an auto-response in Chinese, or
know the difference between real or spoofed DSNs?  Of course not.

All that can be determined from user feedback metrics is that 'X' number
of messages from some source were "unwanted".  Filters can be trained at
removing "unwanted" for specific users based upon their feedback,
however this information's usefulness is limited without knowing overall
source volumes, and whether messages might have been either foreign
language auto-responses, or valid DSNs.

Having "unwanted" considered equivalent of "spam-trap" feedback will
create a number of complaints, especially when dealing with a diverse
range of users.  Spam-trap email-addresses do not solicit email and are
good at excluding auto-responses and valid DSN.  This allows spam-trap
feedback be applied over a larger range of users with less regard to
overall source volume.  Unfortunately, since no provider is able to
ensure all customers are not 0wned, even "spam-trap" feedback can
benefit when overall volume is shared.

Our process consolidates metrics for all activity seen from each
specific source and evaluates related metrics in a non-linear fashion.
Offering this information within ARF feedback would not entail a sizable
increase in report size to include a field of "source activity rate" as
a method to share the number of message events (good or bad) seen over
some number of seconds.

Volume metrics added to an ARF report might look something like:

IP-Activity: [source-IP], [#messages], [#seconds];
DKIM-Activity: [d=], [#messages], [#seconds];

Such as
IP-Activity: 192.0.2.128,34566,86400;
DKIM-Activity: example.com,4013,86400;

When metrics are passed through a feedback conduit, each source should
be qualified as representing spam-trap feedback, or user feedback, where
feedback metrics from multiple sources would be consolidated into at
least two separate categories.

-Doug

---

From: Seth <[email-masked]>
Date: Thu, 17 Dec 2009 15:06:05 -0500 (EST)

> If you're going to do that, why do you need two buttons?

We don't know that it _will_ be the worst case.  To the extent there
are _some_ users who can tell the difference (and the admin can tell
the difference between them and other users), having two buttons gives
better information.

Seth

---

From: Seth <[email-masked]>
Date: Fri, 18 Dec 2009 00:00:19 -0500 (EST)

Douglas Otis <[email-masked]> wrote:

> Agreed. After all, can users recognize an auto-response in Chinese,

If it's to me in Chinese, it's spam, whether or not it's an
auto-response.  Unless, that is, the response is from abuse@ or
postmaster@ a Chinese domain I complained about spam from.

Seth

---

From: Dave CROCKER <[email-masked]>
Date: Thu, 17 Dec 2009 21:08:27 -0800

> the difference between them and other users), having two buttons gives

unless there are _some_ users who are more confused by the choice.  UI 
complexity is not reduced by having more buttons; reducing complexity leads to 
better UI, particularly for the mass market.

as nathaniel notes, these types of design choices for UI design require testings.

for folks who do serious UX work, they do not guarantee that their suggestions 
are right, merely that they are worth testing.

d/
-- 

   Dave Crocker
   Brandenburg InternetWorking
   bbiw.net

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 18 Dec 2009 09:15:21 +0100

Dave CROCKER wrote:
> complexity is not reduced by having more buttons; reducing complexity 
> leads to better UI, particularly for the mass market.

For homogeneity of behavior, I'd note that the buttons resulting in 
sending a message out --Reply, Forward-- open a dialog that shows 
the recipient(s) and the content being sent.

For TiS buttons, older discussions seem to indicate that we don't 
want users to type text. (Therefore, well aimed complaints intended 
for manual evaluation would still go via the Forward button.) 
However, the dialog could sport checkboxes or radiobuttons set up 
according to, say, presence of List-Unsubscribe or 
Authentication-Results headers. Users may adjust these flags before 
hitting Send.

If we want to feed Bayesian filters, we need a spam/ham button.

If messages classified as junk by client filters are automatically 
reported as spam, this circumstance should be flagged on the 
outgoing abuse reports.

> as nathaniel notes, these types of design choices for UI design require 

Yes, and I expect users behavior under traditional MUAs will be 
somewhat different from that of web interfaces users. I'd also 
expect behavior to vary widely among different mail domains.

---

From: ram <[email-masked]>
Date: Fri, 18 Dec 2009 14:16:12 +0530

> output for a "spam?/not-spam?" opinion.

The risk of misreporting doesnt mean that we dont accept reports at
all. 
The MUA should have such easy to use buttons , and the mailserver must
have a protocol to accept the "TIS" complaints

The server may decide to send an ARF report or not. The server should
send the report only if the mails is authenticated ( DKIM or SPF etc ) 
and the mails have valid X-Abuse-To: headers

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 19 Dec 2009 11:26:57 +0100

> input more than output directions.

This thread has become quite intricate. Many useful ideas, but they 
are difficult to sort out because there's little concordance on 
foundations... As a sort of kōan, let me ask this deliverability 
question:

  What are the differences between being vouched for by an MGRS and
  relaying through an external smart host?

---

From: Ian Eiloart <[email-masked]>
Date: Sun, 20 Dec 2009 12:09:09 +0000

--On 17 December 2009 13:55:09 -0500 Paul Russell <[email-masked]> wrote:

>> Anyway, the worst case scenario is that users hit one of the two buttons
>> at  random. In that situation, you simply treat the buttons as
>> equivalent, and  take the same action for each.
>>
>

I'm not going to do that - it's a worst case scenario. I don't believe it's 
necessary with my users. However, some people have such low opinion of 
their users that they might well wish to do that.

The point is that if your users have a mail client with two buttons, it's 
capable of making a distinction that will be useful to some admins. Other 
admins my chose not to make use of the distinction, but if you extinguish 
the information before it reaches the admin, then it's impossible to 
reconstruct it.

---

From: Jose-Marcio Martins da Cruz <[email-masked]>
Date: Sun, 20 Dec 2009 13:54:14 +0100

> as nathaniel notes, these types of design choices for UI design require 

I strongly agree.

Quoting a research paper from Gordon Cormack [1] :

"When explicitly asked to classify messages, human subjects have been reported to exhibit error 
rates of 3%-7% [30, 16]. Tacitly derived labels, such as those obtained from a “report spam” button, 
where it is assumed that unreported messages are ham, may have even higher rates."

The references 30 and 16 in this paper comes from experiments with real data and real subjects 
(SpamOrHam - Graham-Cumming and Hotmail - Yih/Kolcz)

These error rates are most of the time bigger than what can be achieved by spam filters. So it's 
probably a bad idea to consider that user feedback is reliable. User interface shall be as simple as 
possible.

[1] Cormack, G. V. and Kolcz, A. 2009. Spam filter evaluation with imprecise ground truth. In 
Proceedings of the 32nd international ACM SIGIR Conference on Research and Development in 
information Retrieval (Boston, MA, USA, July 19 - 23, 2009). SIGIR '09. ACM, New York, NY, 604-611. 
http://plg.uwaterloo.ca/~gvcormac/cormacksigir09-spam.pdf

JM

> 
> for folks who do serious UX work, they do not guarantee that their 
> suggestions are right, merely that they are worth testing.
> 
> d/

-- 
  ---------------------------------------------------------------
  Jose Marcio MARTINS DA CRUZ           http://j-chkmail.ensmp.fr
  Ecole des Mines de Paris
  60, bd Saint Michel                      75272 - PARIS CEDEX 06
  mailto:[email-masked]

---

From: Seth <[email-masked]>
Date: Sun, 20 Dec 2009 14:24:23 -0500 (EST)

Jose-Marcio Martins da Cruz <[email-masked]>
wrote:

> "When explicitly asked to classify messages, human subjects have
> been reported to exhibit error rates of 3%-7% [30, 16]. Tacitly
> derived labels, such as those obtained from a “report spam”
> button, where it is assumed that unreported messages are ham, may
> have even higher rates."

The error rates differ when the user selects a range of messages for
the button and when the user selects single messages; the former
generates much higher error rates.

Seth

---

From: Jose-Marcio Martins da Cruz <[email-masked]>
Date: Sun, 20 Dec 2009 20:44:07 +0100

> generates much higher error rates.

Probably. Do you have references about that ? Comparative error estimates ?

--

---

From: John Leslie <[email-masked]>
Date: Sun, 20 Dec 2009 18:21:04 -0500

>  relaying through an external smart host?

   The biggest difference is that you can use multiple vouching services
but only one "smart host" at a time.

   Thus, when a vouching service screws up, no action is required, MTAs'
reputation services will probably automatically check your other vouching
services and discount the one that screws up.

   There are other substantial differences in that a vouching service
can continue reporting problems to you in a useful format even while you
_don't_ meet their vouching criteria, which greatly simplifies your
clean-up process.

   For most of us, the most likely clean-up problem is botted computers;
and being able to convice a user that their computer _really_is_ botted
is not necessarily an easy task...

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 21 Dec 2009 11:31:19 +0000

> "When explicitly asked to classify messages, human subjects have been
> reported to exhibit error rates of 3%-7% [30, 16]. Tacitly derived
> labels, such as those obtained from a “report spam” button, where it
> is assumed that unreported messages are ham, may have even higher rates."
>
> The references 30 and 16 in this paper comes from experiments with real
> data and real subjects (SpamOrHam - Graham-Cumming and Hotmail -
> Yih/Kolcz)
>
> These error rates are most of the time bigger than what can be achieved
> by spam filters. So it's probably a bad idea to consider that user
> feedback is reliable. User interface shall be as simple as possible.

But wait, we're talking about messages that the spam filter hasn't 
rejected. Additional data *has* to be useful.

The false positive rates are only a problem if the admin stupid enough to 
consider a single report as definitive. If you deliver a message to 100 
users, and three report it as spam, then you probably take no action. If 20 
report it as spam, then you need to take a closer look.

I certainly don't think a 7% error rate is enough to determine that users 
should not be given the opportunity to distinguish between unwanted mail 
and reportable junk.

You can also combine reporting rates with your bayesian content analyser or 
spamassassin score, or with your reputational score for the sender domain, 
etc.

> [1] Cormack, G. V. and Kolcz, A. 2009. Spam filter evaluation with
> imprecise ground truth. In Proceedings of the 32nd international ACM
> SIGIR Conference on Research and Development in information Retrieval
> (Boston, MA, USA, July 19 - 23, 2009). SIGIR '09. ACM, New York, NY,
> 604-611. http://plg.uwaterloo.ca/~gvcormac/cormacksigir09-spam.pdf
>
> JM
>
>>
> --   ---------------------------------------------------------------

---

From: Ian Eiloart <[email-masked]>
Date: Mon, 21 Dec 2009 12:44:02 +0000

--On 20 December 2009 09:42:03 -0800 Dave CROCKER <[email-masked]> wrote:

>
>
> On 12/20/2009 4:09 AM, Ian Eiloart wrote:
>> The point is that if your users have a mail client with two buttons,
>> it's capable of making a distinction that will be useful to some admins.
>
>
> Only if there is reason to believe the buttons are being used
> consistently and validly and without negative impact.  The mere existence
> of the buttons guarantees neither of the first two.

Think of the user as a voter instead of a judge and jury, when they click 
the "report" button.

Of course, a "block" button is purely an expression of wish - though you 
might want to have them confirm a block, and undo would be useful too.

What we already do know is that many people use "report as spam" when they 
mean "block". Why? Because they have no alternative.

>
> What the mere existence of the buttons DOES guarantee is greater
> complexity to the user interface.
>
> d/

---

From: der Mouse <[email-masked]>
Date: Mon, 21 Dec 2009 09:07:29 -0500 (EST)

> What we already do know is that many people use "report as spam" when
> they mean "block".  Why?  Because they have no alternative.

Not entirely because of that.  At least some of them do not draw a
distinction between spam and other unwanted email.

Of course, these are probably the same people who use "hack" as a
transitive verb in the passive voice.  But a substantial fraction of
the MUA market is made up of them, so any plan that includes
substantial market penetration has to take them into account.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign

---

From: Jose-Marcio Martins da Cruz <[email-masked]>
Date: Mon, 21 Dec 2009 15:32:41 +0100

Ian Eiloart wrote:

...

> rejected. Additional data *has* to be useful.

Sure, I've just mentionned that user feedback isn't reliable. There are many reasons. One thing to
do to mitigate user feedback lack of reliability is to make user interface as "good" as possible.

> 
> The false positive rates are only a problem if the admin stupid enough 
> to consider a single report as definitive. If you deliver a message to 
> 100 users, and three report it as spam, then you probably take no 
> action. If 20 report it as spam, then you need to take a closer look.

Agree, but if 20 report it as spam, but if the other 80 reported it as ham or if they didn't
reported anything these are different situations.

> I certainly don't think a 7% error rate is enough to determine that 
> users should not be given the opportunity to distinguish between 
> unwanted mail and reportable junk.

Sure but... I don't think about a binary decision : using or not user feedback, nor trying to 
correct user feedback till have a perfect clean feedback. The idea behind Cormack paper is to handle 
user feedback the same was as a noisy signal. Try to correct it trivial errors, but accept existence 
of noise and make your system as insensitive as possible to noise.

I made an experiment with some users recently. In this particular experiment, I noticed some obvious 
things : people are concerned about wanted and unwanted mail - ham and spam isn't their problem; 
appreciation for wanted/unwanted messages varies not only from user to user but also depends on the 
moment. The latter means that, at different times, the same messages won't be judged the same way by 
the same user. Also, error rates aren't the same for different kind of messages, even inside the 
same class.

I don't generalize this and, unfortunately, I don't have numbers reliable enough to publish, but I'd 
like to have. I don't believe in opinions which aren't supported by reliable numbers.

> You can also combine reporting rates with your bayesian content analyser 
> or spamassassin score, or with your reputational score for the sender 
> domain, etc.

Sure, but even after that, you still won't have a perfectly clean user feedback.

--

---

From: Seth <[email-masked]>
Date: Mon, 21 Dec 2009 10:46:31 -0500 (EST)

Ian Eiloart <[email-masked]> wrote:

> The false positive rates are only a problem if the admin stupid
> enough to consider a single report as definitive. If you deliver a
> message to 100 users, and three report it as spam, then you probably
> take no action. If 20 report it as spam, then you need to take a
> closer look.

Typical results from known spam (e.g. make penis fast) are that about
1% of users report it.

Seth

---

From: Nathaniel Borenstein <[email-masked]>
Date: Mon, 21 Dec 2009 12:46:00 -0500

On Dec 20, 2009, at 7:09 AM, Ian Eiloart wrote:

> The point is that if your users have a mail client with two buttons, it's capable of making a distinction that will be useful to some admins. Other admins my chose not to make use of the distinction, but if you extinguish the information before it reaches the admin, then it's impossible to reconstruct it.

At the risk of beating a dead horse, this may or may not be true.  The key question isn't whether the mail client can make the distinction, it's whether the users can.  If the users can't use the client's two buttons with sufficiently low error rates, then the resulting data can't possibly be useful to the admins.  In other words, what  you state as fact is actually a testable proposition (about which I am skeptical).  -- Nathaniel

---

From: Matthias Leisi <[email-masked]>
Date: Mon, 21 Dec 2009 19:24:06 +0100

Am 21.12.09 18:46, schrieb Nathaniel Borenstein:

> distinction, it's whether the users can.  If the users can't use the
> client's two buttons with sufficiently low error rates, then the
> resulting data can't possibly be useful to the admins.  In other

When I was responsible for spamfilter operation at a former job, error
rates of "human spamfilters" were considerably higher than FP rates of
any possible solution. This is not scientific evidence, but it
illustrates it nicely:

It was very important for the CEO of the company not to lose mail. It
was thus decided that his assistant would go through his inbox and spam
folder and use "Mark as Spam" and "Mark as Not Spam" buttons to clean up
things.

This assistant was very diligent, and highly capable at her job.
Nevertheless, she had a surprisingly high error rate -- for every
hundred mails which she marked as (not) spam, she mis-categorized maybe
five to ten mails.

I did not watch the other users as closely, but they had similar error
rates. The feedback loop from the users could never have been used for
automated actions due to the low quality (maybe it would have been
possible with a larger user base and appropriate statistical analysis).
The buttons were still an important source for fine tuning of the filter
and to find emerging trends in spammer behaviour. Nothing more, but
nothing less.

-- Matthias

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 22 Dec 2009 11:26:02 +0000

> Typical results from known spam (e.g. make penis fast) are that about

That must depend a lot on the interface that they have. With the mail 
client I have currently, there's no way to know how to report spam. If the 
client had a "report as spam" button, then I'll bet the reporting rate 
would go up.

I get a much lower than 1% reporting rate, except when it comes to spear 
phishing, where the rate is a bit better than 1%, even though people have 
to find out how to do the reporting.

Also, about 90% of the reports I get include minimal headers, so they're 
not very useful.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 22 Dec 2009 11:30:35 +0000

> any possible solution. This is not scientific evidence, but it

I nice story. It says that we must not block mail on a single report, 
especially a report from a different user. That's fair enough, but it's not 
what anyone is suggesting.

The current question is how can we make it easier for users to report spam 
to administrators. The question of what administrators do with the reports 
is secondary - though it might inform the kind of information that they 
want to gather.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 22 Dec 2009 11:40:24 +0000

--On 21 December 2009 06:23:44 -0800 Dave CROCKER <[email-masked]> wrote:

>
>
> On 12/21/2009 4:44 AM, Ian Eiloart wrote:
> One more time:  You cannot be certain of the user interface choices that
> will work well without experimentation.  Good theoretical backing merely
> helps to suggest good UI design ideas, but it does not guarantee that
> they are correct.
>
> Worse, you are trying to assert user needs based on a simple information
> model, rather than a cognitive model, which is quite different.
>
> d/

I've contacted the HCI people at my institution to find out whether they'd 
be interested in doing some research on this topic. I suspect the answers 
might be different for different mail clients, depending on the existing 
interface clutter, and the GUI tools with which they're built.

---

From: Ian Eiloart <[email-masked]>
Date: Tue, 22 Dec 2009 11:45:22 +0000

> distinction between spam and other unwanted email.

I agree. But nobody with a single button is capable of making that 
distinction when reporting. I find it a hard distinction to make on 
Twitter, but I still try. It's an important point to consider when 
responding to the reports.

---

From: Daniel Feenberg <[email-masked]>
Date: Tue, 22 Dec 2009 09:32:59 -0500 (EST)

On Tue, 22 Dec 2009, Ian Eiloart wrote:

> The current question is how can we make it easier for users to report spam to 
> administrators. The question of what administrators do with the reports is 
> secondary - though it might inform the kind of information that they want to 
> gather.
>
>

Whether you want one or two buttons depends upon whether you have one or 
two actions to take. If you have only individual spam filters, then you 
only need one button, call it the "unwanted - unsubscribe" button which 
would attempt to unsubscribe where suitable headers were present, and 
would block otherwise. If you have both individual and global filters, 
then you need both "unwanted - unsubscribe" and "spam" buttons. And you 
probably want to inspect the "spam" items before adding them to any global 
filter.

If I review my own decisions later, I find that about 1% of the 
time I call something I want spam, but I am much more likely to fail to 
call spam as spam. In any case the error percentage must be calculated as 
a percentage of mail, not a percentage of reports to be usefull.

As to the accuracy of user decisions, if you give users only one button, 
they are likely to be less accurate than if they have two buttons. since
they will have no opportunity to select the correct button, they will 
likely select the wrong one rather than do nothing at all.

Daniel Feenberg

---

From: Michael Thomas <[email-masked]>
Date: Tue, 22 Dec 2009 09:45:25 -0800

Ian Eiloart wrote:
> client I have currently, there's no way to know how to report spam. If 
> the client had a "report as spam" button, then I'll bet the reporting 
> rate would go up.
>
> I get a much lower than 1% reporting rate, except when it comes to 
> spear phishing, where the rate is a bit better than 1%, even though 
> people have to find out how to do the reporting.
>
> Also, about 90% of the reports I get include minimal headers, so 
> they're not very useful.
Yet just about everybody has a junk button. Typically it feeds into some 
baysian stuff right now.
There's no reason that in the back end of the mua it couldn't send 
reports somehow, like, say,
an extension to imap or something like that. And then do what Nathaniel 
was saying: treat the
junk button as a "I don't want" button and figure out what that really 
means for the filters.

And it's not like this sort of thing is anything new anyway: lots of 
vendors have "report as
spam" widgets that get bolted onto the side of your favorite MUA. A 
little standardization
would be nice though as it would decouple that UI hassle from the actual 
job of filtering.

Mike

---

From: Nathaniel Borenstein <[email-masked]>
Date: Tue, 22 Dec 2009 14:54:07 -0500

On Dec 22, 2009, at 12:45 PM, Michael Thomas wrote:
> 
> And it's not like this sort of thing is anything new anyway: lots of vendors have "report as
> spam" widgets that get bolted onto the side of your favorite MUA. A little standardization
> would be nice though as it would decouple that UI hassle from the actual job of filtering.

Absolutely -- the report-spam UI will almost certainly be better if it's integrated with the MUA and agnostic regarding the spam engine receiving the report.  The only major open question I'm hearing is how much information that report should contain.  Clearly it should be no more than the number of bits that the user himself can be relied on to provide, where our differing opinions might be resolved via user studies.

It might also be worth considering offering 1 button to most users, but 2 buttons to users who understand the distinction well enough to change a default in their MUA in order to get 2 buttons instead of 1.  I conjecture that the users who would take that action would have a much lower error rate than the average user.   In that scenario, most users would send back a single bit "unwanted" message, but sophisticated users could send back two (or even more?) types of "unwanted" message.  That might be the cleanest data we could hope to get. -- Nathaniel

---

From: Michael Thomas <[email-masked]>
Date: Tue, 22 Dec 2009 12:56:34 -0800

Nathaniel Borenstein wrote:
> Absolutely -- the report-spam UI will almost certainly be better if 
> it's integrated with the MUA and agnostic regarding the spam engine 
> receiving the report.  The only major open question I'm hearing is how 
> much information that report should contain.  Clearly it should be no 
> more than the number of bits that the user himself can be relied on to 
> provide, where our differing opinions might be resolved via user studies.
>
> It might also be worth considering offering 1 button to most users, 
> but 2 buttons to users who understand the distinction well enough to 
> change a default in their MUA in order to get 2 buttons instead of 1. 
>  I conjecture that the users who would take that action would have a 
> much lower error rate than the average user.   In that scenario, most 
> users would send back a single bit "unwanted" message, but 
> sophisticated users could send back two (or even more?) types of 
> "unwanted" message.  That might be the cleanest data we could hope to 
> get. -- Nathaniel
I think the problem is that if you open it up to more than one bit, it 
begs the question of what the
actual number of bits such a button is. I'd say that it's probably got a 
lot of bits -- far more than is
likely that any user could be bothered with.

Want/don't want is nice in its simplicity, and I suspect it's about as 
much as you can expect from users.
However, there's probably a lot of data that MUA's have at their 
disposal to see how you react to mail.
Like, oh say, educing the duration that you viewed a piece of mail. Or 
whether you replied or forwarded.
Or whether you have a habit of deleting particular kinds of them 
en-mass, and other kinds of behaviour
based data.

I think that if we stopped with this absolutist campaign of  "spam/ham" 
(most of us are not on some
paladin's quest  against the evils of spam, after all) and focused more 
on the context sensitive job of
prioritizing mail, we'd all be a lot better off.

Mike

---

From: John Levine <[email-masked]>
Date: 23 Dec 2009 01:53:00 -0000

>begs the question of what the actual number of bits such a button is.

ARF has a bunch of options added at the request of ESPs intended to
support multiple buttons, e.g., spam, opt-out from this list, opt-out
from everything, etc.

As far as I can tell, the only ones that have ever been used are spam
and virus, the latter done mechanically when the message has the bit
pattern of an executable.

Most people have only the dimmest understanding of opt-in, opt-out,
what they subscribed to, etc.  There are a lot of obvious analogies to
paper mail and such that are totally misleading, but close enough to
confuse people.  One button is plenty.

I also think that most of the useful info from multiple buttons could
be extracted mechanically from the mail stream, e.g., if the sender has
sent multiple messages before and it has a List-Unsubscribe: header,
it's probably an opt-out, not a spam complaint.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 31 Dec 2009 19:27:56 +0100

>> provide, where our differing opinions might be resolved via user studies.

Different mailbox providers may process reports in widely different 
ways. While UI's behavior is agnostic, its configuration may refer to 
specific functionalities that the underlying spam engine may or may 
not support.

>> get.

Sophisticated users may choose a mailbox provider according to what 
functionalities it offers. Conversely, ISPs may offer different 
functionalities in order to attract different kinds of users. A good 
MUA should support them all.

> likely that any user could be bothered with.

Assuming that the MUA will use ARF, the number of bits that can be 
stuffed in there is defined by that format and its extensions.

> much as you can expect from users. [...]

"Ham/spam" is not any better than "wanted/not-wanted", as far as 
coherence is concerned. "Legitimate/abusive, w.r.t. applicable privacy 
laws" would perhaps be clearer.

Rather than campaigns, it is a question of functionalities. Servers 
who support mail priority should be able to use reports for that sake. 
Ditto for block or unsubscribe. MGRSs might provide for one additional 
functionality: ban the sender from sending for an amount of time 
proportional to the number of complaints. I'd consider the latter an 
exercise of direct judicial process.