Subject: Re: [marf] MAAWG's Best Practices for Feedback Loops

From: Alessandro Vesely <[email-masked]>
Date: Wed, 01 Sep 2010 11:57:48 +0200

On 31/Aug/10 22:52, Yakov Shafranovich wrote:
> http://www.maawg.org/sites/maawg/files/news/MAAWG_Complaint_Feedback_Loop_BCP_2010-08.pdf
>
> Haven't read it yet

I've given a look through it.  The BCP is very adherent to its "C", 
for "current".  IMHO it may be worth considering possible evolutions, 
since much of this stuff is currently just being standardized.  Some 
specific points:

While the glossary states

   Message Originator — The Sender, but not necessarily the author or
   creator, of a message.

in the "Benefits of Providing Feedback" it is stated that

   The End User’s complaint makes its way back to the Message
   Originator (not necessarily the message Sender, who may be a
   Spammer), allowing the originator to take appropriate action.

That seems confusing to me.  Further instruction about the destination 
are given in "Creating Reports"

   Feedback Loops are usually (but not always) keyed to the "last hop"
   IP address (i.e., the IP address that passed the unwanted message
   to the Mailbox Provider's servers). Consequently, the Feedback
   Provider must be able to process the header from each complaint to
   determine the IP address for the complaint.

In the above snippet, the meaning of "keyed" is not well specified. 
Apparently, the implication is that complaints about forwarded 
messages will not have effect.  The second sentence might mention 
using a database of recently delivered messages, possibly more 
reliable in case users' complaints originate on their MUAs.

The BCP could as Mailbox Providers to consider publishing a list of 
(consenting) Feedback Consumers.  Consent for publishing would be an 
additional bit of consumer information.

In the "Terms of Use" it is stated that

   Data provided in Feedback Messages are provided to a specific,
   approved entity. Information should not be transmitted outside of
   the intended, approved Recipient.

That explicitly forbids relaying.  A mechanism for hierarchically 
forwarding complaints by sender's IP, first RIR allocation to subnet, 
is not considered --it is not "current".

"Application Website" assumes that there will always be a restricted 
number of Feedback Providers, so that discovering them and manually 
apply is a job that mailbox providers can do.  It is silently assumed 
that a few senders apply anyway ("dozen").  OTOH, while it is 
mentioned that Feedback Providers may work for multiple Mailbox 
Providers, it is not stated whether they should accept mass 
subscriptions.  ReturnPath, for one, does not, thus relegating medium 
and small mailbox providers in a different limbo.  IMHO, that is not fair.

Finally, the BCP mentions the practice of delivering to bulk folders, 
but provides no means for notifying this fact to Feedback Consumers. 
It recommends that senders derive such datum using poor heuristics. 
In its wording:

   So if one sees a large difference between a Complaint rate as
   reported by a Mailbox Provider or other reputation system and the
   rate calculated from raw delivery numbers, it is important to look
   closely at the data.

Is that an acceptance that mail delivery is a game?

---

From: SM <[email-masked]>
Date: Wed, 01 Sep 2010 18:07:23 -0700

Hi Yakov,
At 05:34 PM 8/31/2010, Yakov Shafranovich wrote:
>I am not sure IETF needs to have change control, it may be sufficient
>to republish this as BCP RFC via the IETF liason channels.

It's unlikely that the document would be published as a BCP if the 
IETF does not have change control.

Regards,
-sm

---

From: "J.D. Falk" <[email-masked]>
Date: Thu, 2 Sep 2010 11:47:07 -0700

On Sep 1, 2010, at 6:07 PM, SM wrote:

> It's unlikely that the document would be published as a BCP if the IETF does not have change control.

Right, I understand it'd be Informational instead.  I assume it could still be included in standards as a non-normative reference, like other Informational documents...would that be sufficient to fulfill the MARF charter?

---

From: "J.D. Falk" <[email-masked]>
Date: Thu, 2 Sep 2010 12:03:37 -0700

On Sep 1, 2010, at 2:57 AM, Alessandro Vesely wrote:

> That seems confusing to me.  Further instruction about the destination are given in "Creating Reports"
> 
> In the above snippet, the meaning of "keyed" is not well specified. Apparently, the implication is that complaints about forwarded messages will not have effect.  The second sentence might mention using a database of recently delivered messages, possibly more reliable in case users' complaints originate on their MUAs.

I'm not aware of any existing complaint feedback loop which attempts to analyze Received: headers from outside of its own ADMD.  I suppose that's theoretically possible, but it sounds iffy.

DKIM is covered in an appendix.

> The BCP could as Mailbox Providers to consider publishing a list of (consenting) Feedback Consumers.  Consent for publishing would be an additional bit of consumer information.

What would that list be used for?

> That explicitly forbids relaying.  A mechanism for hierarchically forwarding complaints by sender's IP, first RIR allocation to subnet, is not considered --it is not "current".

That's not an IETF "SHOULD NOT" -- but even if it were, it's not a normative requirement.  A complaint-forwarding hierarchy is entirely possible, but it hasn't been tried yet, so there's no current practice to document.

> "Application Website" assumes that there will always be a restricted number of Feedback Providers, so that discovering them and manually apply is a job that mailbox providers can do.  It is silently assumed that a few senders apply anyway ("dozen").  OTOH, while it is mentioned that Feedback Providers may work for multiple Mailbox Providers, it is not stated whether they should accept mass subscriptions.  ReturnPath, for one, does not, thus relegating medium and small mailbox providers in a different limbo.  IMHO, that is not fair.

Today, current practice, most Feedback Providers operate a web site with information about the feedback system.  Tens of thousands of entities (some bulk senders, some ISPs, some I have no idea) have successfully navigated this process.  My reporting-discovery draft will offer an alternative discovery method, but it's not finished and nobody has tried it yet, so it doesn't belong in a BCP.

I don't understand the "mass subscriptions" objection.  Most feedback system application forms (including ours) accept ranges of IP addresses.  Or are you asking for something different?

> Finally, the BCP mentions the practice of delivering to bulk folders, but provides no means for notifying this fact to Feedback Consumers.

Also documenting current practice -- and historical practice, for as long as there've been bulk/spam/junk/quarantine folders.

> Is that an acceptance that mail delivery is a game?

No, it's acceptance that the originator has no visibility into what happens to a message after 250.

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 03 Sep 2010 12:07:03 +0200

On 02/Sep/10 21:03, J.D. Falk wrote:
>>  Apparently, the implication is that complaints about forwarded messages will not have effect.  [...How about] a database of recently delivered messages, possibly more reliable in case users' complaints originate on their MUAs.
>

"Received" can always be forged.  While all MTAs have a mail.log, it 
is seldom practical to use it for automatically checking whether a 
message had really been delivered to a user by the mailbox provider 
who is handling her complaint.  The BCP covers (and recommends) TiS 
buttons on MUAs, but fails to address this check.

> DKIM is covered in an appendix.

Ooops, I missed it.  Appendix 2 is rather neat; IMHO, it should be 
referenced from the "Creating Reports" section, e.g. as
#                                           vvvvvvvvvvvvvvvv
   Feedback Loops are usually (but not always, see Appendix 2) keyed
   to the "last hop" IP address (i.e., the IP address that passed the
   unwanted message to the Mailbox Provider's servers).

>>  The BCP could as[k] Mailbox Providers to consider publishing a list of (consenting) Feedback Consumers.  Consent for publishing would be an additional bit of consumer information.
>

A (small) mailbox provider may add some score to messages from a 
sender listed by a trusted Feedback Provider, and may "auto-subscribe" 
such sender in case their messages are reported as spam later on. 
Such a list would have a meaning similar to VBR.

> Today, current practice, most Feedback Providers operate a web site
> with information about the feedback system.  Tens of thousands of
> entities (some bulk senders, some ISPs, some I have no idea) have
> successfully navigated this process.  My reporting-discovery draft
> will offer an alternative discovery method, but it's not finished
> and nobody has tried it yet, so it doesn't belong in a BCP.

OK

> I don't understand the "mass subscriptions" objection.  Most
> feedback system application forms (including ours) accept ranges of
> IP addresses.  Or are you asking for something different?

Yes, apologies for my poor English.  Say a Feedback Provider operates 
on behalf of hundreds of thousands of Mailbox Providers.  Then, a 
consumer may want to subscribe to all of those FBLs by filling out a 
single form.  Smallish Mailbox Providers would consent to trust the 
checks carried out by the Feedback Provider.  However, the role of 
such kind of Feedback Provider would terminate there, as Feedback 
Messages would be actually sent by each Mailbox Provider.

Again, covering such scenario would be beyond "current" practices. 
Possibly, your reporting-discovery draft may consider it.  Still, the 
BCP might mention that the existing mechanisms are somewhat unfair to 
small Mailbox Providers, and that alternatives are being studied.

>>  Is [delivering to bulk folders] an acceptance that mail delivery is a game?
>

Yeah, in rfc5321's words, "dropping mail without notification of the 
sender is permitted in practice."  Sigh for our lost 821 :-/

---

From: Shmuel (Seymour J.) Metz <[email-masked]>
Date: Fri, 03 Sep 2010 07:18:17 -0400

In <[email-masked]>, on
09/02/2010
   at 12:03 PM, "J.D. Falk" <[email-masked]> said:

>I'm not aware of any existing complaint feedback loop which attempts
>to analyze Received: headers from outside of its own ADMD.  I suppose
>that's theoretically possible, but it sounds iffy.

Having written code that does such an analysis, I have to agree that
it's dicey. You have to keep updating the code as you discover new
Received formats, and you have the dual problems of detecting forgery
and recognizing legitimate handoffs with unexpected trace information.

Based on my experience, I would expect such an analysis to be a
heuristic whose results were to be evaluated by the Mark I eyeball.

-- 
     Shmuel (Seymour J.) Metz, SysProg and JOAT
     Atid/2        <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

---

From: "J.D. Falk" <[email-masked]>
Date: Fri, 3 Sep 2010 14:44:10 -0700

On Sep 3, 2010, at 3:07 AM, Alessandro Vesely wrote:

> "Received" can always be forged.  While all MTAs have a mail.log, it is seldom practical to use it for automatically checking whether a message had really been delivered to a user by the mailbox provider who is handling her complaint.  The BCP covers (and recommends) TiS buttons on MUAs, but fails to address this check.

Probably because all of the authors involved assumed everyone already knows Received headers can be forged.

> A (small) mailbox provider may add some score to messages from a sender listed by a trusted Feedback Provider, and may "auto-subscribe" such sender in case their messages are reported as spam later on. Such a list would have a meaning similar to VBR.

Bypassing the usual signup process?  See appendix 3.  *grin*

But in any case, it's not current practice, so it doesn't belong in a BCP.

> Yes, apologies for my poor English.  Say a Feedback Provider operates on behalf of hundreds of thousands of Mailbox Providers.  Then, a consumer may want to subscribe to all of those FBLs by filling out a single form.  Smallish Mailbox Providers would consent to trust the checks carried out by the Feedback Provider.  However, the role of such kind of Feedback Provider would terminate there, as Feedback Messages would be actually sent by each Mailbox Provider.

Hmm, sounds like an entirely new kind of relationship between entities -- introducing entirely new complexities.

> Again, covering such scenario would be beyond "current" practices. Possibly, your reporting-discovery draft may consider it.  Still, the BCP might mention that the existing mechanisms are somewhat unfair to small Mailbox Providers, and that alternatives are being studied.

Are there other spam-related systems where it's possible to sign up for a whole bunch of unrelated things in one click?

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 04 Sep 2010 11:39:14 +0200

On 03/Sep/10 23:44, J.D. Falk wrote:
>> "Received" can always be forged.  While all MTAs have a mail.log,
>> it is seldom practical to use it for automatically checking
>> whether a message had really been delivered to a user by the
>> mailbox provider who is handling her complaint.  The BCP covers
>> (and recommends) TiS buttons on MUAs, but fails to address this
>> check.
>
> Probably because all of the authors involved assumed everyone
> already knows Received headers can be forged.

Yes, and I wish to thank Shmuel for neatly confirming this and related 
difficulties.

It should then be recommended to dump SMTP transactions onto some 
database.  AFAIK, this is not common practice.  To wit, although 
controversial data retention directives apparently require it (e.g. 
European 2006/24/EC requires CDRs for 6-24 months,) an indexed 
transaction log does not come out-of-the-box with popular MTA software 
packages.

>> sender listed by a trusted Feedback Provider, and may
>> "auto-subscribe" such sender in case their messages are reported
>> as spam later on. Such a list would have a meaning similar to
>> VBR.
>

Of course, it is preferable if the Feedback Consumer has been 
"discovered" by some other means.  Even then, a 3rd party vouch of 
such discovery may help the Feedback Provider wannabee making the 
right decision.

> But in any case, it's not current practice, so it doesn't belong in a BCP.

Agreed.  Still, the "consent for publishing the Feedback Consumer's 
domain name" bit might belong there, as a datum that the Feedback 
Provider may ask and maintain in case they eventually decide to 
actually publish such list.  Or does it hurt to even consider this 
possibility?

>> Again, covering [the massive subscription] scenario would be
>> beyond "current" practices.  Possibly, your reporting-discovery
>> draft may consider it.  Still, the BCP might mention that the
>> existing mechanisms are somewhat unfair to small Mailbox
>> Providers, and that alternatives are being studied.
>
> Are there other spam-related systems where it's possible to sign up
> for a whole bunch of unrelated things in one click?

Not in my limited knowledge.  They need not be completely "unrelated", 
though; for example, they might all be using the same reporting 
software...

---

From: John Levine <[email-masked]>
Date: 5 Sep 2010 23:37:00 -0000

>Are there other spam-related systems where it's possible to sign up for
>a whole bunch of unrelated things in one click?

Sure.  Sign up on any online sweepstakes web site and you'll get more
different kinds of spam than you can imagine.

Or is that not what you had in mind?

R's from Toulouse,
John

---

From: "J.D. Falk" <[email-masked]>
Date: Wed, 8 Sep 2010 12:21:51 -0700

On Aug 31, 2010, at 3:45 PM, J.D. Falk wrote:

> On Aug 31, 2010, at 1:52 PM, Yakov Shafranovich wrote:
> 
>> See:
>> 
> There's still a big open question: MAAWG doesn't want to hand change control over to the IETF without some idea of what kinds of changes this WG will want to make.
> 
> I doubt the changes would be particularly drastic, given the sizable overlap of participants between the groups.  Your thoughts?

So, beyond new ideas (some of which are quite intriguing, but are far from Current or even Emerging Practices), are there any thoughts on how this MAAWG document could/should inform MARF WG efforts?