Subject: Re: [ietf-smtp] You can't hurt a computer's feelings

From: Randall Gellens <[email-masked]>
Date: Sun, 3 Mar 2013 06:17:49 -0800

At 6:36 PM -0500 2/28/13, John R Levine wrote:

>  It'd be interesting.

Here's the answer I got:

At 3:05 PM -0500 3/1/13, David Ross wrote:
>  It would depend on how many connections you have allocated to 
> regular SMTP and how many to authenticated SMTP.
>
>  One EIMS server I have in a small business has both set to 20. 
> Server has been up since Feb 23 and the max connections to date are 
> 7 regular and 4 authenticated at any one time. And with a limit of 
> 20 on the regular SMTP connections I've never seen it hit 20 for at 
> least 5 or more years.
>
>  I suspect this is not as much of an issue now that computers can 
> more easily handle a lot of connections and spammers are likely 
> better at not flooding any one server with connections. I suspect 
> the later also depends on the number of domains hosted.
>
>  Most of the heavy attacks I see these days come in waves where they 
> fire off from 100 or so IPs in a /24 each with a different domain 
> as the sending entity. Spread over a few hours with under 5 
> connection attempts per IP.
>
>  The office I mentioned with 2 domains and 20 people sees one or two 
> of these a day.

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly selected tag: ---------------
...All repairs tend to destroy the structure [of the software], to
  increase the entropy and disorder of the system.  Less and less
  effort is spent on fixing original design flaws; more and more is
  spent on fixing flaws introduced by earlier fixes.  As time passes,
  the system becomes less and less well-ordered.  Sooner or later the
  fixing ceases to gain any ground.  Each forward step is matched by a
  backward one.  Although in principle usable forever, the system has
  worn out as a base for progress.  [...] Systems program building is
  an entropy-decreasing process, hence inherently metastable.  Program
  maintenance is an entropy-increasing process, and even its most
  skillful execution only delays the subsidence of the system into
  unfixable obsolescence.
                             --Fred Brooks in _The Mythical Man Month_

---

From: "John R Levine" <[email-masked]>
Date: 3 Mar 2013 11:22:08 -0500

That matches my experience.  Whatever problem might once have been 
addressed by limiting the number of connections per source, it doesn't 
exist any more.

My server hits its overall limit of 60 all the time when it's getting a 
lot of spam, but it's never more than a handful of those 60 from a single 
IP.

R's,
John

>>  It would depend on how many connections you have allocated to regular SMTP 
>> and how many to authenticated SMTP.
>>
>>  One EIMS server I have in a small business has both set to 20. Server has 
>> been up since Feb 23 and the max connections to date are 7 regular and 4 
>> authenticated at any one time. And with a limit of 20 on the regular SMTP 
>> connections I've never seen it hit 20 for at least 5 or more years.
>>
>>  I suspect this is not as much of an issue now that computers can more 
>> easily handle a lot of connections and spammers are likely better at not 
>> flooding any one server with connections. I suspect the later also depends 
>> on the number of domains hosted.
>>
>>  Most of the heavy attacks I see these days come in waves where they fire 
>> off from 100 or so IPs in a /24 each with a different domain as the sending 
>> entity. Spread over a few hours with under 5 connection attempts per IP.
>>
>>  The office I mentioned with 2 domains and 20 people sees one or two of 
>> these a day.

---

From: Alex Bobotek <[email-masked]>
Date: Sun, 3 Mar 2013 16:22:42 -0800

I'm still aware of high-concurrency attacks on large (i.e., 100M+ subscriber) domains, which admittedly is a small number of domains.  This includes attempts to open too many concurrent connections from a single IP to a single domain.  Most of these are from targeted spam/DoS attacks on a single domain or perhaps a handful of domains launched from a small number of compromised servers sitting in data centers.  This connection limiting keeps a single non-BLed IP address which is obviously up to some anomalous activity from consuming too much server resource.  Most bots are on some BL, and can be efficiently ignored without connections limits.  IMHO, throttles/limits continue to be useful in large enterprise environments, especially if tuned by sender reputation.  

Regards,

Alex
-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of John R Levine
Sent: Sunday, March 03, 2013 8:22 AM
To: Randall Gellens
Cc: [email-masked]
Subject: Re: [ietf-smtp] You can't hurt a computer's feelings

That matches my experience.  Whatever problem might once have been addressed by limiting the number of connections per source, it doesn't exist any more.

My server hits its overall limit of 60 all the time when it's getting a lot of spam, but it's never more than a handful of those 60 from a single IP.  

R's,
John

>> Server has been up since Feb 23 and the max connections to date are 7 
>> regular and 4 authenticated at any one time. And with a limit of 20 
>> on the regular SMTP connections I've never seen it hit 20 for at least 5 or more years.
>>
>> better at not flooding any one server with connections. I suspect the 
>> later also depends on the number of domains hosted.
>>
>> fire off from 100 or so IPs in a /24 each with a different domain as 
>> the sending entity. Spread over a few hours with under 5 connection attempts per IP.
>>

---

From: Randall Gellens <[email-masked]>
Date: Sun, 3 Mar 2013 16:14:14 -0800

At 11:22 AM -0500 3/3/13, John R Levine wrote:

> addressed by limiting the number of connections per source, it 
> doesn't exist any more.

I misstated the EIMS restrictions in my original message.  When I 
checked the documentation, it turns out that EIMS does not enforce 
restrictions on number of connections from the same source.  Instead, 
EIMS has a three-level budget: total number of incoming connections; 
number reserved for hosts in the OK state (having passed 
graylisting); and number reserved for whitelisted hosts.  I apologize 
for mischaracterizing the restrictions (I blame faulty recollection 
and being too lazy to check before writing).

>  My server hits its overall limit of 60 all the time when it's 
> getting a lot of spam, but it's never more than a handful of those 
> 60 from a single IP.

When its getting a lot of spam, does it crowd out legitimate email? 
That's the intent of the EIMS restrictions -- to ensure that some 
connections remain available for hosts that have a higher liklihood 
of being legitimate.

>>> Server has been up since Feb 23 and the max connections to date 
>>> are 7 regular and 4 authenticated at any one time. And with a 
>>> limit of 20 on the regular SMTP connections I've never seen it 
>>> hit 20 for at least 5 or more years.
>>>
>>>   Most of the heavy attacks I see these days come in waves where 
>>> they fire off from 100 or so IPs in a /24 each with a different 
>>> domain as the sending entity. Spread over a few hours with under 
>>> 5 connection attempts per IP.
>>>
>>>   The office I mentioned with 2 domains and 20 people sees one or 
>>> two of these a day.
>
>
>  Content-Type: application/pkcs7-signature; name="smime.p7s"
>  Content-Description: S/MIME Cryptographic Signature
>  Content-Disposition: attachment; filename="smime.p7s"
>
>  Attachment converted: TiLand:smime 199.p7s (    /    ) (00865A81)

-------------- Randomly selected tag: ---------------
Who tells the stories of a culture really governs human behavior.  It
used to be the parent, the school, the church, the community.  Now it's
a handful of global conglomerates that have nothing to tell, but a
great deal to sell.
    --George Gerbner, dean of Annenberg School of Communication

---

From: "John R Levine" <[email-masked]>
Date: 3 Mar 2013 20:00:57 -0500

> When its getting a lot of spam, does it crowd out legitimate email? That's 
> the intent of the EIMS restrictions -- to ensure that some connections remain 
> available for hosts that have a higher liklihood of being legitimate.

Sometimes, but per-IP limits would be useless since the spam comes from 
IPs all over the place.  I haven't adjusted the limit in a long time, 
despite switching to a faster server, so if it mattered I could probably 
increase it to 100.  But in my application, it doesn't.

I collect most of the spam for analysis, which makes my tradeoffs somewhat 
different from other people's.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
"I dropped the toothpaste", said Tom, crestfallenly.

---

From: Randall Gellens <[email-masked]>
Date: Sun, 3 Mar 2013 18:25:19 -0800

At 8:00 PM -0500 3/3/13, John R Levine wrote:

>>  When its getting a lot of spam, does it crowd out legitimate 
>> email? That's the intent of the EIMS restrictions -- to ensure 
>> that some connections remain available for hosts that have a 
>> higher liklihood of being legitimate.
>
>  Sometimes, but per-IP limits would be useless since the spam comes 
> from IPs all over the place.

Right, as I said in my correction, the EIMS limits are not per-IP but 
rather three levels of connection budget: maximum total inbound SMTP 
connections; connections from hosts in the OK state (passed 
graylisting); connections from whitelisted hosts.  So, for example, 
if you have these set to, say, 20/4/2, then of the 20 total allowed 
inbound connections, four are always available for OK hosts, two 
connections remain always available for whitelisted hosts, and up to 
16 can be consumed by hosts that you haven't classified.

-------------- Randomly selected tag: ---------------
Quit smoking on the anniversary of your birth, not
the day of your death.            --Corky Peterson

---

From: hector <[email-masked]>
Date: Sun, 03 Mar 2013 22:36:26 -0500

On 3/3/2013 9:25 PM, Randall Gellens wrote:
> graylisting); connections from whitelisted hosts.  So, for example, if 
> you have these set to, say, 20/4/2, then of the 20 total allowed 

Do you have Connection Sharing (CS) considerations for the white listed 
channels?

I saw how these were consuming connections and CPU time thus increasing 
the potential to reach the connection limits.   The client using CS 
would hold the connection after the first transaction for an extended 
period which technically has 5 minutes to steal from the server, 
increasing the session time mostly wasted in holding and not performing 
any additional transactions.  This causes the attack waves to reach the 
load limit threshold and increased logging of force drops was observed. 
The solution was to drop the normal 5 minutes idle time to about 30 
seconds (might even be smaller) after the first transaction was completed.

--
HLS

---

From: Glenn Anderson <[email-masked]>
Date: Mon, 4 Mar 2013 22:24:22 +1300

On 4/3/2013, at 1:14 ᴘᴍ, Randall Gellens <[email-masked]> wrote:

> I misstated the EIMS restrictions in my original message.  When I checked the documentation, it turns out that EIMS does not enforce restrictions on number of connections from the same source.  Instead, EIMS has a three-level budget: total number of incoming connections; number reserved for hosts in the OK state (having passed graylisting); and number reserved for whitelisted hosts.  I apologize for mischaracterizing the restrictions (I blame faulty recollection and being too lazy to check before writing).

EIMS also has restrictions on the number of connections from the same IP, it is an older setting that I didn't remove when I added the newer priority based limits, it can still be handy for blocking some trivial DoS attacks.

I agree that a limit per IP is pretty much irrelevant if a bot-net is trying to send you spam, and that is definitely a far more common problem these days. On a number of occasions I've seen more than 60,000 unique IPs over 24 hours repeatedly trying to connect to one of my servers that would normally handle ~100 legitimate messages a day. Reserving some connections for recently seen well behaved hosts and for whitelisted hosts definitely helps legitimate mail to get through under those circumstances.

Glenn.