Subject: [dmarc-ietf] AutoForward problems

From: "Douglas E. Foster" <[email-masked]>
Date: Wed, 2 Sep 2020 07:33:38 -0400

For mailing lists, we have pushed the limits of authorization.   But there is another class of problems where sender authorization is not feasible:   mail which is auto-forwarded after a spam-filter has made content-altering changes.

In some respects, the problem is similar to a mailing list.   This mailing list adds a prefix to the subject, adds a footer to the body, and converts content to text.   The first two changes are potentially reversible, while the last one is not.   Commonly used spam filters may add a prefix to the subject, a header or footer to the body, and rewrite embedded URLs to provide time-of-click protection.    Again, the first two changes are potentially reversible, while the last is more difficult.

Because an auto-forward has the potential for senders from any source, third-party authorization of the forwarding domain is not possible.   Instead the mechanism requires either that the recipient domain provides a DMARC exception for the forwarding stream, or the forwarding domain must remove the spam-filter's changes so that signature integrity is restored.

In general, the changes provided by a spam filter are intended for the subscribed client, so it is an interesting licensing question whether those protections should extend to external non-subscriber addresses that are served via auto-forwarding.   But for our purposes, removing those edits would solve the DMARC problem by restoring signature integrity.   So I think we need to re-examine rollback options

I think it is useful to categorize the types of edits that may occur:

- A simple insertion of new text.   
A change log which documents the start point and length of insertion would be sufficient to identify the added text separately from the prior text.   If the document is signed after the changes are applied, an MUA could potentially separate the contents of each author, with author identification.   This may actually enhance efforts to use spam-filter additions as a trust indicator.   An MTA could evaluate the original content against the original signature, or could remove the added content to restore the original message.
- A replacement or removal of text, where the original content can be represented in a message header.    In this case, the change log would include the starting point of the change, the length of the new text, and the entirety of the original text.    An MTA could reverse the changes or evaluate the original signature after applying a virtual reversal of the changes.   An MUA would probably not be able to identify the changed regions to the user, especially if the changes were in the non-visible portion of an HTML component.
- A replacement or removal of text, where the original content is too complex to be stored in the message header.   In this case, the original content can only be recovered if the original content is retained external to the message in a proprietary data store.    Since many organizations use the same vendor for both inbound and outbound gateways, external data stores are plausible.

In the absence of such techniques, the forwarding system (or the outbound gateway) needs to be aware that a message has been forwarded after content-altering changes.   When this is the case, the message either requires a known exception at the receiving system or a From address rewrite so the message will pass DMARC.   At minimum, our DMARC specification should make this clear.

Doug Foster

---

From: Jesse Thompson <[email-masked]>
Date: Thu, 3 Sep 2020 07:41:31 -0500

> For mailing lists, we have pushed the limits of authorization.   But there is another class of problems where sender authorization is not feasible:   mail which is auto-forwarded after a spam-filte…

Yes, this is an increasing problem, as exemplified by the number of enterprises that have started tagging subjects and bodies with [External] warnings.  Coupled with DMARC adoption, the ability for users to auto-forward successfully is shrinking.  

I realize that John's message in the other thread probably wasn't referencing auto-forwarding, but I think his point dovetails to the auto-forwarding issue:

> As always, as I hope we all remember DMARC alignment doesn't mean not spam,
> and you still do all of the stuff you do to sort your mail.  This scheme
> depends on the forwarders you authorize being well-behaved.  That's why I
> am concerned that senders need to be selective about who they allow to
> forward.

I am concerned with:

a) It assumes that the domain owner has the ability and desire to authorize every potential forwarder, even though auto-forwarding is a user-level decision.

b) It assumes that all potential forwarders check for authorization before forwarding -- essentially the same problem with the way most ESPs will use a domain in the From header without checking for authorization via DMARC.

c) Selectively allowing forwarding at the user level is difficult because users are gonna do what they wanna do (you try telling faculty they can't forward).  It's not the case that all enterprises want to prohibit all of their users from auto-forwarding (even though that's the answer you may get if you survey the CISO community)
     
d) Selectively forwarding based on the knowledge of whether the forwarded message will reach the destination is challenging because the intermediary doesn't really know if it will succeed, and selectively forwarding in this way will lead to the user only seeing a subset of their messages being forwarded.  This is essentially what's happening today (see my first sentence in this message), and I know users who are still bound and determined to auto-forward and they have just resigned themselves to the fact that they have to periodically check the intermediary mailbox for all of the messages that didn't survive the forward.

> In the absence of such techniques, the forwarding system (or the outbound gateway) needs to be aware that a message has been forwarded after content-altering changes.   When this is the case, the m…

I agree with this, however, I'm increasingly of the mindset that since this is a user-level issue, we should be thinking about solving it via OAuth mechanisms.  Modern email services do support the ability to fetch from a mailbox, authorized via OAuth, so it might be easier to convince the downstream mailbox providers to enhance their fetching mechanisms to support the features that people expect of inbound mail (i.e. apply inbound filtering rules).  The mailbox provider would still need a way to determine the original DMARC results, spam analysis, etc, but wouldn't that be easier to implement since the message is "frozen in time" and wasn't subjected to the altering changes implied by forwarding?

Jesse

---

From: "Doug Foster" <[email-masked]>
Date: Thu, 3 Sep 2020 17:33:30 -0400

OAUTH vs. More careful forwarding
--------------------------------------------
Pursuing both techniques makes the most sense.    Some users may be
unwilling (or not allowed) to store credentials in the target server, while
being unwilling to do a manual operation to obtain their new messages.

Since OAUTH is a pull operation instead of a push, it provides several
benefits:
- Certainty that the forwarding target actually wants the message.
- Near-certainty that the message will be accepted by the client system,
since the pull bypasses the SMTP filtering process.   This includes both
sender authentication issues like DMARC as well as other content filtering
applied by the forwarding target's domain.

You point out the organizational politics will affect the process:
- For consumer-focused services like Gmail and Hotmail, the user owns the
data and has primary control over its disposition.
- For centralized organizations (most large companies), the organization
owns the data and can control its disposition.
- For decentralized organizations like your university, control is
effectively shared and negotiated.

Forwarding Control Mechanism
--------------------------------------
With any of the above political structures, I argue that the user should not
have sole control over the forwarding process.  

Assume that UserA@DomainA wants to forward to UserB@DomainB:

DomainA interests include:
- If DomainB objects to the messages for any reason, it may blacklist
DomainA and cause disruption to traffic other than this one user's messages.
- The forwarding action may violate company confidentiality or
organizational obligations to protect privacy of others (GDPR, PCI DSS,
HIPPA, etc.)
- DnmainA may want to assess whether the forwarding request is in good
faith, and that it is not a mule account for exfiltration of data.

DomainB and UserB@DomainB interests include:
- Does UserB actually want these messages?
- What if UserB is the victim of a typo?
- What if the auto-forward is being implemented as an attack vector on
UserB?
- What if the auto-forward is going to a non-existent account because of a
typo?

DomainA would benefit from registering the forwarding configuration with
DomainB:
- So that if DomainB has more aggressive spam filtering, the blocked traffic
will be blamed on the source and not on DomainA.
- So that DMARC verification failures can be overlooked.
- More generally, so that DomainB does  not perceive UserA@DomainA as an
open relay or other threat vector.
- If the account is overquota or closed, DomainA has an interest in
receiving notification of this event.

Consequently, I think the protocol should be:
- UserA@DomainA requests forwarding from his domain owner.
- DomainA requires a confirming email from UserB@DomainB, which should be a
trivial effort for UserA@DomainA to make happen.
- DomainA decides how much to engage the domain owner of DomainB for
purposes of registration and whitelisting.
- Assuming that no veto is received, DomainA activates the forward.

Outbound gateways can / should be used to enforce a domain's controls on the
forwarding process.

This is very different from what I have seen implemented in mail store
products.   Many systems allow any user to forward anywhere.   Some systems
allow the domain administrator to disable all forwarding from any account.
I do not think I have seen admin-controlled selective forwarding, but others
have more product experience than I do.

Doug Foster

-----Original Message-----
From: dmarc [mailto:[email-masked]] On Behalf Of Jesse Thompson
Sent: Thursday, September 03, 2020 8:42 AM
To: [email-masked]
Subject: Re: [dmarc-ietf] AutoForward problems

On 9/2/20 6:33 AM, Douglas E. Foster wrote:
> For mailing lists, we have pushed the limits of authorization.   But there
is another class of problems where sender authorization is not feasible:  
mail which is auto-forwarded after a spam-filter has made content-altering
changes.

Yes, this is an increasing problem, as exemplified by the number of
enterprises that have started tagging subjects and bodies with [External]
warnings.  Coupled with DMARC adoption, the ability for users to
auto-forward successfully is shrinking.  

I realize that John's message in the other thread probably wasn't
referencing auto-forwarding, but I think his point dovetails to the
auto-forwarding issue:

> As always, as I hope we all remember DMARC alignment doesn't mean not 
> spam, and you still do all of the stuff you do to sort your mail.  
> This scheme depends on the forwarders you authorize being 
> well-behaved.  That's why I am concerned that senders need to be 
> selective about who they allow to forward.

I am concerned with:

a) It assumes that the domain owner has the ability and desire to authorize
every potential forwarder, even though auto-forwarding is a user-level
decision.

b) It assumes that all potential forwarders check for authorization before
forwarding -- essentially the same problem with the way most ESPs will use a
domain in the From header without checking for authorization via DMARC.

c) Selectively allowing forwarding at the user level is difficult because
users are gonna do what they wanna do (you try telling faculty they can't
forward).  It's not the case that all enterprises want to prohibit all of
their users from auto-forwarding (even though that's the answer you may get
if you survey the CISO community)
     
d) Selectively forwarding based on the knowledge of whether the forwarded
message will reach the destination is challenging because the intermediary
doesn't really know if it will succeed, and selectively forwarding in this
way will lead to the user only seeing a subset of their messages being
forwarded.  This is essentially what's happening today (see my first
sentence in this message), and I know users who are still bound and
determined to auto-forward and they have just resigned themselves to the
fact that they have to periodically check the intermediary mailbox for all
of the messages that didn't survive the forward.

> In the absence of such techniques, the forwarding system (or the outbound
gateway) needs to be aware that a message has been forwarded after
content-altering changes.   When this is the case, the message either
requires a known exception at the receiving system or a From address rewrite
so the message will pass DMARC.   At minimum, our DMARC specification should
make this clear.

I agree with this, however, I'm increasingly of the mindset that since this
is a user-level issue, we should be thinking about solving it via OAuth
mechanisms.  Modern email services do support the ability to fetch from a
mailbox, authorized via OAuth, so it might be easier to convince the
downstream mailbox providers to enhance their fetching mechanisms to support
the features that people expect of inbound mail (i.e. apply inbound
filtering rules).  The mailbox provider would still need a way to determine
the original DMARC results, spam analysis, etc, but wouldn't that be easier
to implement since the message is "frozen in time" and wasn't subjected to
the altering changes implied by forwarding?

Jesse

_______________________________________________
dmarc mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/dmarc

---

From: Jesse Thompson <[email-masked]>
Date: Thu, 3 Sep 2020 17:38:17 -0500

> OAUTH vs. More careful forwarding

OAuth, as opposed to basic authentication, does not require the user to share their password with a 3rd party.  The OAuth access token is a credential, but it's individualized to the app using it, and can be revoked.

The manual operation only comes into play during authentication (and whatever session length requirements are imposed by the provider).  I'd say the manual effort is a wash, and the opportunity to introduce re-authentication at periodic intervals improves security and prevents forgotten configurations from persisting.

> effectively shared and negotiated.

Right, and the organization can (or should be able to) configure policies to manage this shared relationship; potentially allowing for consumer-focused services to move a bit in one direction, and centralized organizations to move a bit in the other.

> Forwarding Control Mechanism

We do implement some controls over who can forward, and where those users can forward, and who can tell which users where they can/cannot forward.  It's all custom, of course.  I'm unsure how common it is.

The UX comes into concern because when the user is forwarding out of the system they are disengaged with that system.  How does the system communicate to the user that some new policy is in effect short of sending an email or stopping the mail flow and hoping they notice and call for support?  

With the fetch model, at least there are more opportunities to interact with the user where they are (i.e. during the authorization flow during re-authentication after a session timeout).  Of course, all of that depends on the fetching side's UI for handling re-authentication, so it's not a slam dunk alternative to SMTP forwarding.

Jesse

---

From: "John Levine" <[email-masked]>
Date: 3 Sep 2020 22:43:37 -0000

In article <[email-masked]>,
Jesse Thompson  <[email-masked]> wrote:
>> and you still do all of the stuff you do to sort your mail. ...

As you say, it's the same problem, it's what people see as the same
message but with changes that fail with current authentication
schemes.

>a) It assumes that the domain owner has the ability and desire to authorize every potential forwarder, even though
>auto-forwarding is a user-level decision.

It's not the domain owner, it's more likely the MTA deciding what signatures to apply.

>c) Selectively allowing forwarding at the user level is difficult because users are gonna do what they wanna do (you
>try telling faculty they can't forward).  It's not the case that all enterprises want to prohibit all of their users
>from auto-forwarding (even though that's the answer you may get if you survey the CISO community)

Yeah. The likely damage from allowing wisc.edu to forward seems pretty
low, the damage from outlook.com or gmail.com considerably more.

R's,
John
-- 
Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly

---

From: "Douglas E. Foster" <[email-masked]>
Date: Thu, 3 Sep 2020 22:05:24 -0400

Although I re-raised the change log issue as part of the auto-forward problem, I am hoping that it will have significant benefit to the mailing list community also.

Of the three types of content changes that I proposed, the easiest to specify and get implemented is the first type, where the mediator only adds content, adds a change log indicating the additions, and signs the result.   I am hoping and assuming that if mailing lists have freedom to add their branding to the subject and body, most lists would not need to make more complex changes.   

The signed change log would allow participating recipients to identify the signed additions added by the list or other mediator, while also identifying the signed original after the list additions are virtually removed.   Once the additions and the original are reliably identified to a source domain, suspicion of spoofing is no longer a concern.  Each chunk of content can be evaluated based on the reputation of the verified source domain and the specifics of the content.

Nested additions are possible.    Each new signature adds an entry to a verification stack.   Any change can be removed, virtually or actually, by reversing the change at each level, working backwards from last to first.   In practice, the stack is expected to be short.  To prevent malicious misuse, the specification should put a small limit on the number of layers in the stack.

Just as importantly, I believe it solves the problem of letting the list know whether From rewriting is necessary or not.   I argue that there is no information-leakage risk associated with a domain publishing a DNS entry that says "This domain understands multi-author documents of type 1".    This type of flag says nothing about what the domain will do with any particular message with multiple authorship.  The recipient domain will have the option of passing the entire message, stripping the wrapper and passing the original, or blocking the whole thing.   But it will not be blocking a message because it does not understand the message's actual identity, which is the cause of our current problem.   Once a list knows that its identity will not be misjudged, it gains the freedom to assume that its content will be judge fairly, so "From" rewriting should not be necessary when sending to domains that have published this flag.

Because this approach adds confidence rather than risk, I am hopeful that even AOL would be willing to implement support for it.

The more complex changes, edits and deletions, require a more complex specification and more complex code.   That level of specification will be needed to provide a solution to spam filters that do URL rewriting.   But that complexity can wait for another day while we try to get a level 1 change log accepted.

John, I am hoping that you find some opportunity here.   Do you?

Doug Foster

----------------------------------------

From: "John Levine" <[email-masked]>
Sent: 9/3/20 6:44 PM
To: [email-masked]
Subject: Re: [dmarc-ietf] AutoForward problems

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Sat, 5 Sep 2020 11:35:47 +0200

On Fri 04/Sep/2020 04:05:24 +0200 Douglas E. Foster wrote:
> 
> Of the three types of content changes that I proposed, the easiest to specify 
> and get implemented is the first type, where the mediator only adds content, 
> adds a change log indicating the additions, and signs the result.   I am hoping 
> and assuming that if mailing lists have freedom to add their branding to the 
> subject and body, most lists would not need to make more complex changes.

The change log must not be a generic patch, but rather a stylized list of 
pre-canned modifications, much like envisaged in the dkim-transform draft. 
This limitation can reduce the attack surface, although it cannot prevent 
malicious URLs in the footer.

> The signed change log would allow participating recipients to identify the 
> signed additions added by the list or other mediator, while also identifying 
> the signed original after the list additions are virtually removed.

I don't think the change log has to be signed.  If undoing the changes leads to 
a verifiable signature, then add a dkim=pass for the original signer.  Else 
dkim=fail.  Signing the change log doesn't hurt, but it doesn't help either.

If verification succeeds, Authentication-Results: can report enough 
transformation details to allow the MDA to restore the original From:, in case 
the MLM rewrote it.

> Once the additions and the original are reliably identified to a source
> domain, suspicion of spoofing is no longer a concern.  Each chunk of content
> can be evaluated based on the reputation of the verified source domain and
> the specifics of the content. >
> Nested additions are possible.    Each new signature adds an entry to a 
> verification stack.   Any change can be removed, virtually or actually, by 
> reversing the change at each level, working backwards from last to first.

I beg to disagree.  On the one hand, we already have ARC to unwind a chain of 
message handlers.  The "defect" of ARC is that it needs a full domain 
reputation system in order to work reliably.  Where the reputation of 
"intermediate" mediators is needed, ARC is the right tool.

On the other hand, a deterministic tool should only be interested in who is the 
actual author of a message and what has the domain owner to say about 
attributing such authorship.  This can be done without assessing reputation.

IMHO, the original author domain deserves an aggregate report mentioning the 
result of evaluating DKIM transformations, even if From: was rewritten.  So 
does the last From: rewriter.  Intermediate mediators don't.

Best
Ale
--

---

From: "Douglas E. Foster" <[email-masked]>
Date: Sat, 5 Sep 2020 13:54:43 -0400

What I am trying to accomplish is different than what can be accomplished with the canned-modifications indicator.    To explain, I need to digress into my theory of operation for spam filters:

1) Source identification allows assignment of a Source reputation.  Source reputation is more important than content filtering, because hostile content always comes from a source that should not be trusted.   Content filtering always produces false positives and false negatives, and the workarounds to those errors are always dependent on source identification

2) Impersonation is always attractive to an attacker because it allows him to exploit the reputation of the impersonated identity.   Therefore impersonation is an inherently untrusted activity.

3) Spam filtering will assign sources to three reputation groups:   negative reputation (rejected), neutral reputation (acceptance depends on content filtering), and positive reputation (some or all content filtering bypassed.)   SPF and DMARC are designed to block impersonation, and mailing lists look like impersonated traffic, so the message moves from neutral to negative reputation.  How to overcome that?

One solution is to use out-of-band information to justify overlooking the negative clues, then implementing local policy based on that informatoin, so that traffic is moved from negative reputation to positive reputation.   ARC and canned-modification tagging are approaches to providing in-message data intended to support application of that local policy.    But we have found no way to ensure the out-of-band information flow needed to justify the local policy, for all of the mediators that need that status.   We have also identified no method for the recipient to notify the mediator that the local policy is established, although this could also be handed out-of-band.  However, these techniques have the benefit of depending on the mediator and the recipient, and not on the sender.

A second solution depends on explicit sender authorization to eliminate the apparent impersonation.   This category encompasses conditional signatures, ATSP, RHSWL, DKIM delegation, and SPF inclusion.   These approaches require the involvement of sender, list, and recipient.   We have concluded that these approaches are victims of unlikely participation by senders, and further limited because the list does not know if a recipient will recognize the sender's authorization.    Finally, I observed that this solution cannot help with the problem created by spam filter tagging prior to an auto-forward, so it cannot solve the whole problem.

A third solution is to abandon DMARC and allow impersonation to be unrestricted.   

I am suggesting a fourth approach.   This one seeks to address the impersonation problem by clearly identifying each part of the message to its source, so that impersonation is not an issue, and each source's contribution is evaluated based on that source's reputation and that source's content.  The goal is to move the imputed source reputation from negative to neutral, rather than from negative to positive.   If the source reputation can be defaulted to neutral, the approach can be used by any mediator without prior registration with recipients and without any prior authorization by senders.   

But on continued reflection, I realize that this approach requires complete isolation between the content added by a mediator and the content provided by the originator.   Any other changes to the original content could maliciously alter the original intent of the author, and an automated spam filter has no ability to identify changes of intent.  So is there a group of mediators who only require the ability to add a subject prefix, subject suffix, body header, or body footer?   

- The better spam filters offer URL rewrite, which alters original content.    The weaker spam filters may only use these four features, but they are the ones that are least likely to add an exotic new feature like dual authorship detection.   So I reluctantly conclude that there is no significant opportunity for using this approach on the "spam filter with auto-forward" problem.

- But is there is a group of mailing lists that only need these four capabilities?   I was hoping so.

DF

From: Alessandro Vesely <[email-masked]>
Sent: 9/5/20 5:36 AM
To: [email-masked]
Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists

Best
Ale
--

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Mon, 7 Sep 2020 13:30:28 -0700

Historically, I've found that a negative source reputation is easy to
dodge.  It's trivial to come from an unknown IP address or register a new
domain name, so an actor with a negative reputation can trivially move to a
neutral one.  Thus, a receiver/verifier seeking to make a meaningful
judgement can only really focus on positive reputations when making
meaningful filtering decisions; everyone else is basically the same in
terms of value.

Another way to look at this: DKIM (and I believe SPF) only really tells you
something interesting when it passes.  That means (for DKIM) the content
was unmodified, and the signature is validated by a key that is verifiably
present in some domain's DNS data.  That means the domain announcing that
key implicitly "takes some responsibility" for the content just verified.
So the only variable here is the value, or reputation, of the verified name.

All of this is orthogonal to DMARC though, which doesn't care about
reputation.  It only cares about alignment, and specifically alignment that
is under complete control of the sender.

-MSK

On Sat, Sep 5, 2020 at 10:55 AM Douglas E. Foster <fosterd=
[email-masked]> wrote:

> What I am trying to accomplish is different than what can be accomplished
> with the canned-modifications indicator.    To explain, I need to digress
> into my theory of operation for spam filters:
>
> 1) Source identification allows assignment of a Source reputation.  Source
> reputation is more important than content filtering, because hostile
> content always comes from a source that should not be trusted.   Content
> filtering always produces false positives and false negatives, and the
> workarounds to those errors are always dependent on source identification
>
> 2) Impersonation is always attractive to an attacker because it allows him
> to exploit the reputation of the impersonated identity.   Therefore
> impersonation is an inherently untrusted activity.
>
> 3) Spam filtering will assign sources to three reputation groups:
> negative reputation (rejected), neutral reputation (acceptance depends on
> content filtering), and positive reputation (some or all content filtering
> bypassed.)   SPF and DMARC are designed to block impersonation, and mailing
> lists look like impersonated traffic, so the message moves from neutral to
> negative reputation.  How to overcome that?
>
> One solution is to use out-of-band information to justify overlooking the
> negative clues, then implementing local policy based on that informatoin,
> so that traffic is moved from negative reputation to positive reputation.
> ARC and canned-modification tagging are approaches to providing in-message
> data intended to support application of that local policy.    But we have
> found no way to ensure the out-of-band information flow needed to justify
> the local policy, for all of the mediators that need that status.   We have
> also identified no method for the recipient to notify the mediator that the
> local policy is established, although this could also be handed
> out-of-band.  However, these techniques have the benefit of depending on
> the mediator and the recipient, and not on the sender.
>
> A second solution depends on explicit sender authorization to eliminate
> the apparent impersonation.   This category encompasses conditional
> signatures, ATSP, RHSWL, DKIM delegation, and SPF inclusion.   These
> approaches require the involvement of sender, list, and recipient.   We
> have concluded that these approaches are victims of unlikely participation
> by senders, and further limited because the list does not know if a
> recipient will recognize the sender's authorization.    Finally, I observed
> that this solution cannot help with the problem created by spam filter
> tagging prior to an auto-forward, so it cannot solve the whole problem.
>
> I am suggesting a fourth approach.   This one seeks to address the
> impersonation problem by clearly identifying each part of the message to
> its source, so that impersonation is not an issue, and each source's
> contribution is evaluated based on that source's reputation and that
> source's content.  The goal is to move the imputed source reputation from
> negative to neutral, rather than from negative to positive.   If the source
> reputation can be defaulted to neutral, the approach can be used by any
> mediator without prior registration with recipients and without any prior
> authorization by senders.
>
> But on continued reflection, I realize that this approach requires
> complete isolation between the content added by a mediator and the content
> provided by the originator.   Any other changes to the original content
> could maliciously alter the original intent of the author, and an automated
> spam filter has no ability to identify changes of intent.  So is there a
> group of mediators who only require the ability to add a subject prefix,
> subject suffix, body header, or body footer?
>
> - The better spam filters offer URL rewrite, which alters original
> content.    The weaker spam filters may only use these four features, but
> they are the ones that are least likely to add an exotic new feature like
> dual authorship detection.   So I reluctantly conclude that there is no
> significant opportunity for using this approach on the "spam filter with
> auto-forward" problem.
>
> ------------------------------
> *From*: Alessandro Vesely <[email-masked]>
> *Sent*: 9/5/20 5:36 AM
> *To*: [email-masked]
> *Subject*: Re: [dmarc-ietf] AutoForward problems - Change log benefits to
> mailing lists
>

---

From: "John Levine" <[email-masked]>
Date: 7 Sep 2020 20:46:46 -0000

In article <CAL0qLwbTcYvf9HKREQi=[email-masked]>,
Murray S. Kucherawy  <[email-masked]> wrote:
>present in some domain's DNS data. ...

Yes, exactly. 

With SPF, no sensible person rejects mail on SPF -ALL other than the
special case of plain -ALL that means it sends no mail whatsoever.
Other than that, neither DKIM nor SPF can distinguish between "this is
fake" and "this was sent by a route that SPF/DKIM can't describe."

---

From: "Doug Foster" <[email-masked]>
Date: Tue, 8 Sep 2020 08:09:25 -0400

SPF/DMARC are not optimal tools for detecting malice.    In my experience, the abundance of sender configuration errors are the limiting factor.

However, I disagree about negative reputation.    Content filtering alone is insufficient and even more error prone.   In the last year, I have had spam campaigns about LED lighting, stand-up desks, touchless thermometers, and knife sharpeners.  I cannot anticipate all the ways that spammers will hide their dirty deeds.   But I know it is spam, not merely unwanted advertising, because of receiving many similar messages from many different domains using many different servers.   Third-party RBLs help but are insufficient.   I am gradually building my own reputation database based on the traffic that I am receiving.   By blocking the problem sources, I have been able to get the spam problem under something approaching good control.   Content filtering is a useful tool for day-zero detection of a new spam source.   Once a source is detected, it needs to be blocked.

Whether a message passes SPF and DMARC criteria is part of my search critieria for unwanted traffic, but definitely not the only one.   As has been observed, actual spoofing of the From address is not a huge part of my problem at present.   This is largely because spammers have easy enough tools in Friendly Name spoofing and corporate logo misuse.   But I also attribute that low volume to the existence of SPF and DMARC. 

Doug Foster

From: Murray S. Kucherawy [mailto:[email-masked]] 
Sent: Monday, September 07, 2020 4:30 PM
To: Doug Foster
Cc: IETF DMARC WG
Subject: Re: [dmarc-ietf] AutoForward problems - Change log benefits to mailing lists

Historically, I've found that a negative source reputation is easy to dodge.  It's trivial to come from an unknown IP address or register a new domain name, so an actor with a negative reputation can trivially move to a neutral one.  Thus, a receiver/verifier seeking to make a meaningful judgement can only really focus on positive reputations when making meaningful filtering decisions; everyone else is basically the same in terms of value.

Another way to look at this: DKIM (and I believe SPF) only really tells you something interesting when it passes.  That means (for DKIM) the content was unmodified, and the signature is validated by a key that is verifiably present in some domain's DNS data.  That means the domain announcing that key implicitly "takes some responsibility" for the content just verified.  So the only variable here is the value, or reputation, of the verified name.

All of this is orthogonal to DMARC though, which doesn't care about reputation.  It only cares about alignment, and specifically alignment that is under complete control of the sender.

-MSK

On Sat, Sep 5, 2020 at 10:55 AM Douglas E. Foster <fosterd=[email-masked]> wrote:

DF

  _____  

On Fri 04/Sep/2020 04:05:24 +0200 Douglas E. Foster wrote:

 

 

 

the MLM rewrote it.

 

 

Best

Ale

--

 

 

 

 

 

 

 

 

dmarc mailing list

[email-masked]

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Tue, 8 Sep 2020 13:30:26 -0700

On Tue, Sep 8, 2020 at 5:09 AM Doug Foster <fosterd=
[email-masked]> wrote:

> However, I disagree about negative reputation.    Content filtering alone
> is insufficient and even more error prone.   In the last year, I have had
> spam campaigns about LED lighting, stand-up desks, touchless thermometers,
> and knife sharpeners.  I cannot anticipate all the ways that spammers will
> hide their dirty deeds.   But I know it is spam, not merely unwanted
> advertising, because of receiving many similar messages from many different
> domains using many different servers.   Third-party RBLs help but are
> insufficient.   I am gradually building my own reputation database based on
> the traffic that I am receiving.   By blocking the problem sources, I have
> been able to get the spam problem under something approaching good control.
>   Content filtering is a useful tool for day-zero detection of a new spam
> source.   Once a source is detected, it needs to be blocked.
>
>
>
> Whether a message passes SPF and DMARC criteria is part of my search
> critieria for unwanted traffic, but definitely not the only one.   As has
> been observed, actual spoofing of the From address is not a huge part of my
> problem at present.   This is largely because spammers have easy enough
> tools in Friendly Name spoofing and corporate logo misuse.   But I also
> attribute that low volume to the existence of SPF and DMARC.
>

Suppose I'm one of your touchless thermometer spammers.  Your system
identifies me and the DKIM signing domain I'm using.  I notice, through
well-established means, that my spam is no longer getting through to you.
So I register a brand new junk domain name, perhaps sadehaiuhfiewn.com or
whatever a few smashes of the keyboard yields, and start signing with that
instead of whatever domain I was using before.  For a couple of bucks, I
have now escaped my negative reputation in your system.  Maybe I bounce it
through a botnet too, so that you can't catch me with an IP reputation
either.

Negative reputations are trivially shed.  It follows that it's not terribly
useful to track them, at least not long-term.  You end up with records of
spammy domains that you'll notice only sent mail for the shortest of time
ranges, long enough to get in undetected or under the guise of "too new to
block", and then abandoned when they stop working.  Blocking domains you've
never heard of before is often disruptive when, say, you join a loyalty
program for some vendor you've never dealt with before and actually do want
their mail, so you're between a rock and a hard place.

Instead, positive reputations are the things on which you can reliably act,
giving such messages preferential treatment.  It's generally a much higher
bar, plus the namespace of domains that manage to earn positive reputations
is small, and they tend to be well-behaved over longer periods of time.

Content filtering is a different matter.  It's focused on what's in the
message, irrespective of where it came from.  But that's a whole new game
to play, and definitely not anything in which DMARC is interested.

-MSK

---

From: Brandon Long <[email-masked]>
Date: Tue, 8 Sep 2020 15:05:40 -0700

On Tue, Sep 8, 2020 at 1:30 PM Murray S. Kucherawy <[email-masked]>
wrote:

> Negative reputations are trivially shed.  It follows that it's not
> terribly useful to track them, at least not long-term.  You end up with
> records of spammy domains that you'll notice only sent mail for the
> shortest of time ranges, long enough to get in undetected or under the
> guise of "too new to block", and then abandoned when they stop working.
> Blocking domains you've never heard of before is often disruptive when,
> say, you join a loyalty program for some vendor you've never dealt with
> before and actually do want their mail, so you're between a rock and a hard
> place.
>
> Instead, positive reputations are the things on which you can reliably
> act, giving such messages preferential treatment.  It's generally a much
> higher bar, plus the namespace of domains that manage to earn positive
> reputations is small, and they tend to be well-behaved over longer periods
> of time.
>

I disagree, we track reputations both good and bad, and they are
incorporated in spam rules across the ladder.  A surprising number of
negative reputations are not shed, even at very-low... and sure, we do
sunset reputations that go unused.  At the very least, there is a time lag
before the spammer notices the effect and switches.

I mean, a blacklist is ultimately a determination that a reputation is so
low as to block completely, and that seems to be the main way that
anti-spam information is distributed and used by most medium to small
providers.

That set of botnet IPs definitely will get a low reputation themselves and
end up on blacklists.

And forcing the spammers to spend money on things like new domain names is
part of the benefit.

OTOH, we also don't believe in "too new to block", unknown reputation is a
great reason to apply throttling at the least.

Maybe some of this is large system stuff, where you can expect to see more
traffic and things don't tend to be unique... but of course we also get
complaints from very small volume folks as well.

Brandon