Subject: [Asrg] 6. Proposals - C/R and "callbacks"

From: Tony Toews <[email-masked]>
Date: Fri, 26 Dec 2003 11:39:39 -0700

At 12:00 PM 2003/12/24 -0500, you wrote:

> >  From my very simplistic view of the email systems it seems to me that
> > the msgid id could be part of the verification mechanism.   The
> > originating MTA stamps the outgoing email with a unique message id.
> > Once the header portion of the email is received then the recipient MTA
> > does a name lookup on the originating MTA and verifies that the
> > originating MTA sent that msgid.  Once the receipient MTA finishes
> > accepting the email the originating MTA then never uses that msgid again.
>...
>
>This approach is outlined by William Elan (he calls it "callback"):
>http://www.elan.net/~william/asrg-emailpathverification-presentation.pdf
>This is also mentioned in the CRI proposal (level 2):
>http://www.ietf.org/internet-drafts/draft-irtf-asrg-cri-00.txt

Thanks for the URLs.  I'll review those

>This basic argument against this is that it is very resource intensive,
>with a lot of extra costs such as more powerful servers that can handle
>callbacks, more bandwidth, and extra disk space to store IDs.

More powerful servers?  Well sure but what are we doing now to handle spam 
with word and Bayesian etc filtering along with RBL lookups?   In addition 
the msgid would only be verified by the receiving MTA when the sending MTA 
is actually sending the email header info.  So the msgid is already in RAM 
thus no cost to do a disk I/O to retrieve information.

More bandwidth?  If this kind of thing cuts spam by a significant margin 
that's a good tradeoff. Also the network traffic is only as many bytes as 
the msgid plus some overhead along with the ack by the transmitting 
MTA.   Along with retrieving the IP address of the sending MTA from the 
domain name in the email header.

Disk space?  Every email currently has, or should have, a unique msgid.

>There are
>also DOS issues: someone forging a domain can cause the forged domain to
>go under with too many callbacks.

DOS could happen today with someone sending lots of emails to a given 
receiving MTA.

>There are privacy issues as well, and
>possible replay attacks.

I don't see the privacy issues here.    Replay attack?  Same as a DOS.

>However, the bottom line here, is that why go through the trouble of
>checking every single message. Granted that it increases costs,
>nevertheless if we can significantly reduce the problem through
>verification of MTAs as opposed to senders and messages, why go through
>the extra costs of message verification. This proposal is on the table,
>but we will pursue it only if a significant advantage vs. costs can be
>proven for this way of doing things, over others requiring less costs.

Ok, this makes sense.   But if I understand the proposals the public key 
will be retained by the DNS servers.   Thus increasing the load on those 
servers.   Mind you they would be cached by your "closest" DNS server so 
that wouldn't be too bad.

Tony
-----
Tony Toews, Microsoft Access MVP
Microsoft Access Links, Hints, Tips & Accounting Systems at
    http://www.granite.ab.ca/accsmstr.htm

---

From: Philip Miller <[email-masked]>
Date: Fri, 26 Dec 2003 16:05:01 -0500

Tony Toews wrote:
> [snip]
> More powerful servers?  Well sure but what are we doing now to handle 
> spam with word and Bayesian etc filtering along with RBL lookups?   In 
> addition the msgid would only be verified by the receiving MTA when the 
> sending MTA is actually sending the email header info.  So the msgid is 
> already in RAM thus no cost to do a disk I/O to retrieve information.

In terms of total computation necessary, sure, Bayesian is more expensive 
than just about anything else. However, keep in mind relative costs: 
bayesian filtering is often happening on the users' desktops, or on a 
private mailserver where the load doesn't affect any public services. Have 
you heard of any large corporations or ISPs setting up individual Bayesian 
filters for each user and rejecting mail at the MX using those filters? I 
certainly haven't.

> that's a good tradeoff. Also the network traffic is only as many bytes 
> as the msgid plus some overhead along with the ack by the transmitting 

Yes, the bandwidth savings would probably be excellent. Keep in mind that 
computation is currently cheaper than storage or anything else. So people 
are willing to expend a lot of computation.
If we could come up with a way to increase the bandwidth costs of sending 
spam without increasing the recipients' costs as well, that'd be really 
cool. Of course, that disregards the loads of spam sent from systems that 
don't cost spammers a thing: hijacked systems, virus/worm infected systems, 
virtualmda systems, etc.

>> There are also DOS issues: someone forging a domain can cause the
>> forged domain to go under with too many callbacks.
> 

Yes, but a callback DoS would mean an extra level of indirection to track 
down the attacker. Look at what happened to grc.com a while back, with what 
its owner, Steve Gibson, called a "Distributed Reflection Denial of 
Service". A whole lot of machines started forging TCP SYN packets to central 
internet routers' BGP ports, and they flooded him. Hopefully, there would be 
some logging on the intermediate reflection MTAs in an attack like this, but 
it wouldn't mean much.

>> There are privacy issues as well, and possible replay attacks.
> 

Privacy issues are not likely if this is done with any sort of care at all. 
We just need to make sure that numbers used can only be related to message 
parameters by the server that generated the numbers.
Replay attack prevention would take some more effort. We'd need to ensure 
that a spammer can't hack a legitimate mailing list server and send spam 
with the same ID numbers as messages sent to the list initially.

> that wouldn't be too bad.

Generally, DNS load arguments haven't really held up, except if DNS servers 
are expected to perform any sort of computations. So the proposal is fine on 
that front.

Philip Miller

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Matthew Elvey <[email-masked]>
Date: Mon, 29 Dec 2003 11:54:27 -0800

On 12/26/2003 1:05 PM, Philip Miller sent forth electrons to convey:

> In terms of total computation necessary, sure, Bayesian is more 
> expensive than just about anything else. However, keep in mind 
> relative costs: bayesian filtering is often happening on the users' 
> desktops, or on a private mailserver where the load doesn't affect any 
> public services. Have you heard of any large corporations or ISPs 
> setting up individual Bayesian filters for each user and rejecting 
> mail at the MX using those filters? I certainly haven't. 

Data point: Yahoo is trying and failing.  To PAYING customers of 
MailPlus ( http://mailplus.mail.yahoo.com/ , it includes SBC DSL and 
dial-up customers) they claim you can "Train your personal spam filters 
to recognize what /you/ consider spam" using the SpamGuard feature.  
Except it doesn't work: when I try to train it on more than a handful of 
spam at a time, it just times out.  Yahoo Level 2 support worked on it 
for a few weeks and claimed it was fixed, but it is still broken (it 
fails to process/delete/train on the spam); they're supposed to be 
working on it again. I did so and I still get about a dozen spam a day 
to my yahoo account. After training on > 1000 spam,  I still get about a 
dozen spam a day to my yahoo account inbox.  Caveat: nothing I've read 
specifically states that spamguard is Bayesian, so it could be something 
else.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 29 Dec 2003 17:13:48 -0500

> to recognize what /you/ consider spam" using the SpamGuard feature.  

  Finland has a large mail redirection system set up, similar to
pobox.  I forget what it's called (ikki?), but it's fairly large.

  Last I heard, they're using SpamAssassin, which is (mostly) keeping
up with the load.  They split the mail into filtered & unfiltered
streams, with the filtered streams going to dedicated machines.  After
filtering, the messages are sent back into the main queue, and
forwarded to the final destination.

  CPU time is relatively cheap, so this works.  Sendmail can handle
large amounts of traffic, with careful design.

  But Bayesian filtering takes large amounts of disk space per use.
I've heard numbers from 1M to 10M, with much of the database touched
during active filtering.  For a system forwarding messages for 10's of
1000's of users, that's expensive.  The disk space isn't the problem,
it's the simultaneous access of 1000's of accounts to reading and
writing 10M databases.

  When coupled with many people using the forwarding service to
subscribe to mailing lists, it's nearly impossible to partition that
load in a way which doesn't increase it.

  Alan DeKok.