[Dshield] Spam blocking validation

Jon R. Kibler Jon.Kibler at aset.com
Mon Aug 21 22:33:04 GMT 2006

Andrew Willy wrote:

I can only state our experience handling email for several companies of 5 to 250 email addresses. I hope it helps.
> 1) Is there data available on spam received by an average mid-size business,
> and how much of that is blocked and how much is delivered to the end user?

Email to valid email addresses:
    This mostly depends on whether the person has ever used their email address for non-business purposes, including ever having used it to post to news groups or register at web sites. Also, another critical factor is whether the email address is published anywhere, especially on a web page.
        Business Only Unpublished Email Addresses: About 30% spam.
        General or Published Email Addresses: About 85% spam.
        Widely Published Email Addresses: About 95% spam.
        User That Registered Their Email Address At A Porn Site: About 99.9999% spam.

Email to all addresses in a domain (valid addresses + dictionary attacks):
    About 3% (and this figure is RAPIDLY dropping!) of all MTA connections result in the delivery of legitimate email. (Some weeks it has been less than 1%.)

How much spam gets through?
    Less than 1% of messages received by the user are spam.

A question you didn't ask: How much malware (excluding phishing) gets through?
    (Knock on wood!) To date, since late 1999, zero. (Again, knock on wood real quick!) However, I think this is most organization's biggest email hole, and the single greatest risk to email. Bottom line, signature based malware detection alone is inadequate.

Another question you didn't ask: How much phishing gets through?
    Less than 5% of phishing attempts.

> 2) What is a realistic expectation of false positives?  We strive for zero,
> as I am sure everyone does, however at what percentage do we know we're
> completely out of tune?

This has to be broken into several categories:
    Non-spam content identified as spam.
       Way less than 0.1%.
    MTAs blocked because of missing or forged host names.
       Probably the single biggest issue: Businesses running MTAs that do not have valid hostnames or have fwd-rev DNS mismatches. (Fix via whitelist.)
    MTAs blocked by RBLs or use ISP hostnames.
       Probably the second biggest issue: Businesses running MTAs from netblocks identified as DHCP addresses, use ISP DSL (cable, etc.) hostnames, or have been identified as spam sources due to earlier compromises.
    Content identified as malware, but is not.
       Rare: Probably the biggest issue is bugtraq XSS postings with sample code getting quarantined.
    Password protected zips.
       Block a lot of malware this way, but I can only think of a few cases of legit files blocked.
    Attachments identified as potential malware, but renamed and passed on to the user.
       Rare: Usually excel or word files with macros.
    HTML tags potentially associated with executable content that are disabled.
       Common: Although I don't really consider this a false positive, probably 10% to 20% of legit HTML email has potentially dangerous tags that we disable (e.g., SCRIPT --> NOSCRIPT). That said, we have never had a complaint.
    (I am sure there are some that I am forgetting.)

The single biggest complaint that we get is that someone's email is being blocked (rejected) because of DNS issues on the sender's end. These result in about 0.05% false positives. (Solution: whitelist)

> 3) We experience waves of spam increase the spam messages a user receives
> exponentially over a few days. I imagine these bursts are due to outbreaks
> of new zombie malware -- is that a reasonable idea?

One possibility. Another is that most spammers don't send continuously... they send in waves. Zombies make it easier and faster to blast out spam when there is a spam run.

At least that is my opinion.

Jon Kibler
Jon R. Kibler
Chief Technical Officer
Advanced Systems Engineering Technology, Inc.
Charleston, SC  USA
(843) 849-8214

Filtered by: TRUSTEM.COM's Email Filtering Service
No Spam. No Viruses. Just Good Clean Email.

More information about the list mailing list