TweetFollow Us on Twitter

Review: SpamAssassin

Volume Number: 19 (2003)
Issue Number: 6
Column Tag: Reviews

Review: SpamAssassin

Regaining Control of your Inbox

by Vicki Brown

Ready, aim, fire!

Separating the Wheat from the Chaff

The popularity of the Internet, plus increases in connectivity (and email access), has given rise to a corresponding avalanche of spam. We all know what spam is - unsolicited junk email offering goods or services that we don't want, don't need, and often didn't want to know existed! We all agree that we would like less of it. Failing that, how can we wade through it, manage it, and delete it without losing our desirable mail (or our minds) in the process?

Many "solutions" have been proposed to deal with the spam problem. These range from anti-spam filters integrated inside email applications (e.g., the junk mail filters in Apple's Mail) to standalone products of various levels of complexity, ease of use, effectiveness, and cost.

My current solution of choice is SpamAssassin (http://www.spamassassin.org), in combination with procmail (http://www.procmail.org). I post-process the results through my desktop email client, (Eudora), for a very powerful and accurate spam-killing solution.

SpamAssassin provides many features, including:

  • user-configurable spam score threshold

  • ability to re-write Subject lines

  • user-configurable filters, using Perl regular expressions

  • modification of existing Spam scores

  • use of statistical, "Bayesian" analysis

  • "auto-learning"

  • optional use of DNS blacklists (e.g., Real-time Blackhole List)

  • optional use of Network Checksum Tests (services that compare message checksums to known spam)

  • "whitelists" (From addresses that are considered OK)

  • Accepted message languages

Note that SpamAssassin and procmail both run on the server side (where mail is originally delivered), not on the client side (i.e., usually not on the desktop) and neither has a GUI front end or a particularly "user-friendly" configuration mode. If you, or your users, are not technically inclined, I recommend that you choose a different path. However, if you enjoy tinkering, have some understanding of regular expressions (e.g., you've used Perl) and like to work with text-based configuration files, you will find the SpamAssassin/procmail combination to be powerful, flexible and, more important, accurate.

Configuration

SpamAssassin comes pre-configured with a large set of tests that it will perform on all incoming mail. In addition, you can add new tests, skip tests, raise or lower the "score" assigned by a given test or, using procmail, cause some mail to skip SpamAssassin altogether. Goto http://www.spamassassin.org/doc.html to see more detailed documentation. You'll even find a pointer, at that URL to a SpamAssassin configuration generator tool, designed to make it easier to customize an installation of SpamAssassin with some common options.

Much of the power of SpamAssassin comes from its configurability, its use of Perl regular expression pattern matching, and its interaction with procmail.

For example, many spammers have started adding ever-changing sets of numbers to the ends of subject lines, to fool the simpler junk mail filters. One GUI-based anti-spam application I tried had a set of filters designed to try to catch mail of this form. The filters looked like this:

   if Subject ends in 0
or if Subject ends in 1
or if Subject ends in 2
...
or if Subject ends in 9

This is rather cumbersome; worse, it doesn't handle the spammers who throw in a space at the very end. SpamAssassin's approach is both shorter and more flexible. This regular expression matches any digit, followed by 0 or more whitespace characters, at the end of the Subject line.

Subject =~ /\d\s*$/

SpamAssassin's interaction with procmail also allows me to specify whether mail is even sent through the SpamAssassin filters. For example, I can specify that mailing list messages should be delivered directly. This procmail recipe checks the message headers for one that matches the given Reply-to string, sending mail from the SpamAssassin-Talk list to my pre-defined default mailbox, without further processing.

:0 H
* ^Reply-to:.*spamassassin-talk
$DEFAULT

Results

SpamAssassin doesn't actually delete any spam. Instead, it tags each piece of mail it processes with a set of headers, e.g.,

X-Spam-Status: Yes, hits=6.2 required=2.5
tests=AWL,CLICK_BELOW,FREE_TRIAL,HTTP_WITH_EMAIL_IN_URL,
spam_PHRASE_05_08,VLB_spam_OFFER_4,VLB_TO_NOT_NAME,WEB_BUGS
version=2.43
X-Spam-Flag: YES
X-Spam-Level: ******
X-Spam-Checker-Version: SpamAssassin 2.43 (1.115.2.20-2002-10-15-exp)

X-Spam-Flag is a simple Boolean; if a message is considered potential spam, the flag is set (and the value is YES). X-Spam-Level provides more information; one "*" is given for each integer value of the spam score. X-Spam-Status provides full scoring information as well as which tests passed. Note that some tests raise the spam score while others are designed to lower it. After all tests have been run, the resulting score is compared to a (user configurable) threshold. If the score exceeds the threshold, the message is tagged as spam.

The user gets to decide how to handle the mail after it's been processed and tagged. One possibility is to continue processing with procmail. For example, the SpamAssassin docs assert that mail with a score of 15 or higher is almost certainly spam (with 0.05% false positives, according to SpamAssassin's rules/STATISTICS.txt file). After some use, I decided to lower this cutoff even more. I have configured procmail to delete any mail tagged with a Spam score of 10 or higher, using the following recipe:

:0:
    * ^X-Spam-Level: \*\*\*\*\*\*\*\*\*\*
    /dev/null

Other than that, all remaining mail is delivered. I have occasionally found false positives in mail with scores of less than 10; I want to check that mail by eye.

Here's where my third tool comes in. I use Eudora as my mail application and make extensive use of Eudora's filter mechanism to post-process my potential spam. I separate potential spam by mailboxes and labeling, based on certain criteria such as whether the message contains my full name (or only my email address), how high the spam score is, and any other interesting criteria. I scan the "Junk" mailboxes a few times a day, pull out false positives (setting new filters to catch those the next time!), and trash the real junk.


Installation

This is only a review, not a tutorial, so the installation section will hand wave a lot. You'll need a server machine running some variant of *nix (e.g., FreeBSD, Linux, or Mac OS X). If your server runs Mac OS X, you'll need to make sure it has been configured to deliver mail locally (e.g., using sendmail or qmail). How to set up a server is beyond the scope of this review.

You should also note that, although SpamAssassin can be run without procmail, its flexibility (and power) increases when it is used in conjunction with procmail. SpamAssassin does not include code to handle local mail delivery; it relies on procmail (or something else) for delivery. Unless you know that a reliable "something else" is available, use procmail.

Procmail may already be on your server; it is pre-installed on most *nix systems these days (use whereis procmail or locate procmail to check, then read the documentation to determine how to configure procmail for your server situation). If it's not pre-installed, download and build procmail from the procmail.org web site.

SpamAssassin is probably not pre-installed on your system; however, installation is simple. In fact, you'll probably be able to install it without first downloading the archive. SpamAssassin is written as a Perl module; the easiest way to install it is by using Perl's CPAN shell (from the command line, as root):

perl -MCPAN -e shell 
o conf prerequisites_policy ask
install Mail::SpamAssassin
quit

Alternatively, you can always download the latest archive from spamassassin.org, then build and install SpamAssassin according to the documentation. If you have any problems, be sure to read the Installation notes on the SpamAssassin site. Check the documentation for more information on configuration, as well as useful options.

Summary

If you're looking for a powerful, flexible, and accurate solution to the spam problem, I recommend that you consider the team of SpamAssassin and procmail. Both programs are free and come with example recipes and suggestions for how to use them to your best advantage. Each has a support community that you can draw upon for assistance and ideas, as well as mailing lists to keep you informed. If you have a server on which to install it (note that SpamAssassin can be installed server-wide or on a per-user basis) and know (or are willing to learn about), regular expressions, SpamAssassin may be the spam-killing tool you've been looking for. Give it a try.

References


Vicki Brown

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Crypt of the NecroDancer Pocket Edition...
Crypt of the NecroDancer Pocket Edition 1.0 Device: iOS Universal Category: Games Price: $4.99, Version: 1.0 (iTunes) Description: Crypt of the NecroDancer is an award winning hardcore roguelike rhythm game. Move to the music and... | Read more »
Gear-grinding puzzle title Inner Circle...
If you saw our post earlier this month announcing the imminent release of ZPlay’s new creation, Inner Circle, you’ll be happy to know that it’s now available on the App Store. Established in 2010, developer and publisher ZPlay have taken the... | Read more »
CSR Racing 2: Your guide to what's...
CSR Racing 2, or CSR2, as it likes to call itself, has finally arrived. The follow-up to the immensely popular drag racing game CSR Racing is the first release from NaturalMotion since the studio's acquisition by Zynga in early 2014. [Read more] | Read more »
Nanuleu (Games)
Nanuleu 1.1 Device: iOS Universal Category: Games Price: $2.99, Version: 1.1 (iTunes) Description: Nanuleu is a strategy game where you take control of ancient magical trees that protect the land from an invading dark force. A... | Read more »
The Slaughter: Act One (Games)
The Slaughter: Act One 1.0.323 Device: iOS Universal Category: Games Price: $3.99, Version: 1.0.323 (iTunes) Description: “The game mixes realism and surrealism to create a story that can cause just as much laughter as fear. A-” -... | Read more »
NEO TURF MASTERS (Games)
NEO TURF MASTERS 1.0 Device: iOS Universal Category: Games Price: $2.99, Version: 1.0 (iTunes) Description: NEOGEO’s legendary golf game is back, in a brand-new mobile version with touch controls! NEO TURF MASTERS (also known as “BIG... | Read more »
How to send money to a friend with the F...
Facebook is already a lot of things to a lot of people. It's a social network, sure, but also a tool for catologing memories, chatting with friends, and much more. [Read more] | Read more »
Branch out into the Chinese market with...
Created a game that you’re super enthused about, only to find yourself struggling to get it the exposure that you believe it duly deserves? With the time, energy and money that goes into development, it’s only right that games should be given the... | Read more »
Pocket Arcade Story (Games)
Pocket Arcade Story 1.00 Device: iOS Universal Category: Games Price: $4.99, Version: 1.00 (iTunes) Description: Here comes a new challenger: it's the arcade simulation game you've been waiting for! Build your very own gaming... | Read more »
How to get coins faster in Rodeo Stamped...
There comes a time in a cowboy or cowgirl's life when all the riding and lassoing skills in the world aren't enough. You're going to need some cold, hard cash to keep your sky zoo expanding in Rodeo Stampede. [Read more] | Read more »

Price Scanner via MacPrices.net

New App Reminds Us to Put Down Our Phones and...
Mode, a new smartphone app that makes us more mindful of how we use our devices, debuts in the app stores today. The Mode app tracks time spent in different modes of day-to-day life without... Read more
ZuumSpeed Personalized Speedometer + HUD For...
RMKapps has announced the release and immediate availability of ZuumSpeed 1.0, its personalized speedometer plus heads up display for iOS devices. ZuumSpeed gives users over 18 custom fonts available... Read more
Apple refurbished clearance 15-inch Retina Ma...
Apple has Certified Refurbished 2014 15″ 2.2GHz Retina MacBook Pros available for $1609, $390 off original MSRP. Apple’s one-year warranty is included, and shipping is free. They have refurbished 15... Read more
9-inch 128GB Silver iPad Pro on sale for $50...
B&H Photo has the 9.7″ 128GB Silver Apple iPad Pro on sale for $699 including free shipping plus NY tax only. Their price is $50 off MSRP. Read more
Why Use Indie Opera And Vivaldi Instead Of Sa...
For many years my web browser workhorses were various permutations and spinoffs of the Netscape/Mozilla/Firefox Open Source platform, and the Norwegian indie browser Opera, which I took a shine to... Read more
Western Digital Launches Worlds Fastest 256GB...
At the Mobile World Congress in Shanghai Western Digital Corporation this week introduced a new suite of 256 gigabyte (GB) microSD cards, which includes the new 256GB SanDisk Extreme microSDXC UHS-I... Read more
KeyCue 8.1 Integrates With Typinator To Displ...
Ergonis Software has released KeyCue 8.1, a new version of the company’s keyboard shortcut cheat sheet. KeyCue 8 introduced a new way to define a wide variety of triggers, which can be used to... Read more
Save up to $600 with Apple refurbished Mac Pr...
Apple has Certified Refurbished Mac Pros available for up to $600 off the cost of new models. An Apple one-year warranty is included with each Mac Pro, and shipping is free. The following... Read more
21-inch 2.8GHz iMac on sale for $1199, save $...
Amazon has the 21″ 2.8GHz iMac (model #MK442LL/A) on sale for $1199.99 including free shipping. Their price is $100 off MSRP, and it’s the lowest price available for this model. Read more
13-inch 2.5GHz MacBook Pro (Apple refurbished...
Apple has Certified Refurbished 13″ 2.5GHz MacBook Pros available for $829, or $270 off the cost of new models. Apple’s one-year warranty is standard, and shipping is free: - 13″ 2.5GHz MacBook Pros... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* iPhone 6s and New Products Tester Ne...
…we therefore look forward to put out products to quality test for durability. Apple leads the digital music revolution with its iPods and iTunes online store, Read more
*Apple* Retail - Multiple Positions, Willow...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.