The Impact of Spam on your Institution
Karl A. Krueger
kkrueger at whoi.edu
Wed Feb 11 19:06:40 GMT 2004
On Tue, Feb 10, 2004 at 09:01:23AM +1100, Tim Lane wrote:
> 1) Is SPAM a non problem, minor problem or major problem in your
> institution? (perhaps incidcate % of spam received of total email).
Spam is a substantial problem for a number of our users. We have seen
increasing call for more aggressive anti-spam techniques. Some users
have even resorted to changing their email addresses without forwarding
(with the disruption that this implies) in order to avoid flood levels
Mostly, though, I see spam for us as a quality of service issue, not a
technical. Our mail system is robust enough that it would not be
overwhelmed by an order of magnitude more email -- but our users don't
want spam! By blocking spam, we are improving the quality of email
service that we offer to our clients.
At the same time our user community has little tolerance for persistent
false positives in spam blocking, so we have to be quick to respond with
whitelisting when messages are reported as incorrectly blocked. (We
whitelist an IP address or sender email address maybe once or twice a
(Please note, "SPAM" in capitals is a trademark of Hormel. "Spam" is
unsolicited bulk email. See http://www.spam.com/ci/ci_in.htm -- the
owners of this trademark have actually been rather pleasant about not
suing all us sysadmin types for trademark dilution.)
> 2) Is or has your institution actively addressing SPAM in a way that has or
> will substantialy reduce the impact of SPAM?
I believe so. Excluding viruses, we block over 4000 spam per weekday,
out of about 35000 messages delivered to or from 1500 email users.
("Excluding viruses" because right now, Mydoom junk is over half of what
we are rejecting -- mostly as "user unknown"!)
> 3) What type of controls (both technical and human) are being used?
We use a mixed-method approach, implemented with Postfix on our mail
1. DNSBLs. We use a number of DNSBLs, including SBL-XBL, SPEWS, AHBL,
BOPM, and ORDB. These provide the vast bulk of effective spam
blocking (excepting viruses), and when combined with responsive
whitelisting do *not* cause significant false positives, contrary to
what some anti-DNSBL folks will tell you.
2. Local blocklist. This is a list of small network ranges from which
our users have received spam, which they have reported to us. This
is actually implemented as a local DNSBL for efficiency, using
Michael Tokarev's rbldnsd program.
3. Regular expression content filters. We filter on a number of common
strings in spam, e.g. spamvertised products, disclaimers referring to
nonexistent laws, the ADV tag in English and Korean. This is also
where we reject Microsoft .EXE attachments and do basic virus
4. SMTP integrity checking. We reject SMTP sessions with unauthorized
pipelining, or which submit an obviously falsified HELO. We find
that a few hundred spam a day can be rejected simply by rejecting
mail from remote sites which HELO with the IP address of our mail
5. SpamAssassin. We pass all incoming mail through SpamAssassin, which
tags "suspected spam" for our clients to filter client-side if they
choose to. We are also implementing Websieve, a Web interface for
users to write Mail Delivery Agent filters for their accounts. Under
the hood, SpamAssassin uses Vipul's Razor and DCC, among other
We also publish Web pages to our user community describing what we do
about spam, how to report spam to us, and how to report false positives.
One of the Web pages is a CGI which can extract the mail log entries for
a given user's mail -- a user can authenticate to it, and see the log
entries for the messages that were blocked as spam or viruses. This, we
find, increases clients' confidence that we are not blocking real mail.
(If anyone wants this CGI, please email me off-list. It is written in
Python and requires Apache mod_python, mod_auth_ldap, and that your user
authentication be done against an LDAP server. It parses syslog entries
in the format written by Postfix 2.0.)
Karl A. Krueger <kkrueger at whoi.edu>
Network Security -- Linux/Unix Systems Support -- Etc.
Woods Hole Oceanographic Institution
More information about the unisog