TweetFollow Us on Twitter

A platform for protecting mail servers.

Volume Number: 20 (2004)
Issue Number: 6
Column Tag: Programming

RAE Internet's MPP

by Lon Baker

A platform for protecting mail servers.

Now I can sleep at night.

The joy of infected and unwanted e-mail

On May 3, 1978, the first spam message was received. In 1981 the Apple Viruses 1, 2, and 3 are seen "in the wild" on the Apple II operating system, spread through pirated computer games, and Texas A&M University. The first threat of email delivering a virus arrived in the form of the "Good Times" email hoax in 1994 with the "Melissa" virus, W97M/Melissa, delivering on the threat in 1999, executing a macro which forwarded itself to 50 people from a users address book. The genie was out of the bottle, and managing e-mail servers has never been the same.

With the growing sophistication of attacks today being the single largest headache for anyone running a mail server endures. The growing drain on time, and resources to battle this scourge has forced every administrator to develop an arsenal of tools and techniques. But how does an administrator manage the array of anti-virus and anti-spam scanners?

Building a e-mail filtering solution

Here at Speedymac, our clients, including technical support services, real estate appraisers, and business owners of all types, demand dependable e-mail with minimal impact from spam and viruses in order to conduct their business. Over time, we explored numerous scanning solution to meet our clients needs. Each solution called for a significant level of work to integrate with our mail server. We encountered daily challenges in managing virus definition updates, installing new filters, and numerous other administrative tasks.

The most time consuming aspect of deploying these solutions has always been determining the most dependable and efficient method for integrating each scanner with the mail server. In some cases a simple perl script was all that was required, while others called for homegrown code pieced together in spite of companies refusing to officially support Mac OS X.

We found this process took far too much development time in light of the evolving sophistication, and volume of the message payloads targeting our clients. We wanted to spend more time improving accuracy, and dependability of our service and less time gluing the pieces together.

RAE Internet has developed a product that fit the bill for what we needed. MPP (Message Processing Platform), a Unix daemon deployed on a mail server, or as a SMTP proxy that provides a platform for integrating a wide range of scanning solutions. Built around the concept of consolidating configuration and management tasks, MPP's architecture supports a wide range of mail servers with scanning tasks handled via dynamic scanning modules.

The mail servers supported by MPP include Communigate Pro, Postfix, Sendmail, SergeMail, and Qmail, and the dynamic scanning modules currently available for Mac OS X include Sophos Anti-Virus, Cybersoft's Pattern Authority, ClamAV open source anti-virus scanner, and the ubiquitous SpamAssassin.

A key point for us was that MPP offered a consistent, and stable platform to implement a wide array of scanning solutions on Mac OS X. We were able to quickly integrate a uniquely powerful solution for our clients through the blending of numerous scanning modules.

We have found that the use of multiple modules significantly enhanced the protection provided to our clients. Using several anti-virus modules, for example, helped catch new virus outbreaks faster than a single scanner, since each anti-virus vendor may have significantly different lag time between the initial outbreak of a virus, the necessary definition being published, and the updated definition reaching our servers.

When it comes to spam filtering, MPP offered us the opportunity to deploy Pattern Authority in addition to SpamAssassin. Obviously, any additional resource for preventing spam is a welcome addition to most server administrators.

What does MPP actually do?

MPP acts as the intermediary in the scanning process, taking messages from the MTA, preparing the message for scanning, then handing the message off to each active Dynamic Scanning Module for processing. When scanning is complete, MPP compares the results against the configuration before signaling the MTA with the correct action to take.

The architecture of MPP is based on a single Unix daemon managing numerous worker processes. The number of worker processes can be configured to handle different message volumes, with each additional worker process spawning any necessary processes needed for each active dynamic scanning module.


Figure 1.

The configuration of MPP is handled via a single text file. A Webmin module, to handle the majority of configuration tasks through a Web Browser, is included, though more advanced configuration options require manual editing via the command line.

A wide range of processing options can be configured based on the results from the scanners. A message may have headers added, be quarantined, rejected, deleted, or passed through based on these settings, as well as actions to take should a scanning error occur.

Messages flagged by the MPP configuration and scanning results can be quarantined, which is common for messages infected with a virus, and MPP provides for a disk-based quarantine for spam messages, in addition to infected messages.

Management of quarantined messages is also handled through the Webmin module. Allowing you to review, delete, forward, and resubmit quarantined messages. The latter is helpful in the event a message is falsely flagged for quarantine.

One of the most important aspects of any scanning solution is maintaining current virus definitions. In this area, MPP offers update scripts for the commercial scanning modules that can be manually controlled through the Webmin interface.

The choice RAE made to support multiple solutions through the use of dynamic scanning modules allows greater flexibility in bringing new engines to Mac OS X server administrators. This helps protect our investment, and encourage additional companies to support Mac OS X through a relationship with RAE potentially.

Deploying MPP

MPP can be deployed on your mail server, or as an SMTP proxy server. If MPP is deployed as an SMTP Proxy, a minimally configured mail server is required to handle the SMTP services, since MPP does not contain its own SMTP server. Some may view this as a drawback, but we believe that it is an advantage, since it allows MPP to focus on what does best.

RAE offers package installers for installing MPP on Mac OS X. Though the installers are quite handy, they are not without their occasional flaws, and it is highly recommended you do test installs on a machine not in production to insure that the installer does not overwrite the existing configuration or license files.

Installation of MPP can vary widely based on which dynamic scanning modules you intend to use. In the case of the commercial scanners, RAE provides native Mac OS X installer packages for both Sophos, and Pattern Authority, while the open source scanners, such as SpamAssassin, and ClamAV, require you to install these solutions separately on the server from their respective sources.

There are a few loose ends when deploying MPP. Automating virus definition updates requires manually creating crontab entries to call the provided update scripts. In the case of ClamAV, you will have to create or use an update script that is provided with the ClamAV source distribution.

Updating an existing MPP installation is fairly straightforward using a simple TAR archive. The one catch being that you must completely stop MPP, and kill any existing processes. This can leave you vulnerable to un-scanned messages passing through to end users while the update is in progress.

The support for a wide range of MTA's and platforms allows an administrator to mix and match as they see fit. In our case, we have our central mail servers running Communigate Pro using Mac OS X Server on collocated Xserves. Our backup mail servers run Postfix, and handle backup MX for our clients using less expensive equipment, and facilities.

Configuring and Managing MPP

Configuration and management of MPP is accomplished through the Webmin module included with the product. Through this tool, most configuration tasks are performed, though a few of the more advanced configuration options require manual editing of the configuration file. The Webmin module also provides a fully functional interface for managing quarantined spam, and infected messages.

MPP configuration options are broken into several distinct sections: anti-virus, anti-spam, and vendor specific options. The configuration of MPP is quite flexible, having both "Scan" and "Do No Scan" options, allowing you to customize which messages are scanned. The drawback is that these options are global, and apply to both anti-virus and anti-spam.

The anti-spam configuration options include basic whitelist and blacklist settings that allow you to designate e-mail addresses, and domains for each list. The main drawback of these configuration options is that they only apply to the "From:" and "To:" headers.

MPP does not manage the configuration of SpamAssassin, so you still have to manage your it's configuration separately. Since SpamAssassin configuration is a world in and of itself, this is not surprising or unwelcome.

When deploying MPP as an SMTP Proxy, the current version does not support managing the configurations of each proxy remotely. You are, therefore, in the position of syncing the configuration files either manually, or via a homegrown solution such as rsync.

MPP is a rapidly maturing product that offers a fundamental choice one has to make. Do you commit to utilizing the product to completely manage the email filtering? Or, do you simply allow it to "tag" messages for processing down stream? This decision is quickly being shaped by the legislative actions revolving around spam and privacy. But it is nice to have the option to determine the method of management depending on your environment.

Performance

As mentioned earlier, MPP is implemented as a native Unix daemon following a daemon/worker process model, which allows MPP to grow as your mail volume increases simply by adding more processes. If mail volume grows beyond the capabilities of a single server with MPP, multiple scanning servers can be deployed in the form of proxy servers. This scaling has the potential to significantly improve performance by distributing the load across numerous servers and networks.

Each active dynamic scanning module contributes to the overall performance, memory requirements, and scanning speed of MPP. As is to be expected, performance impact can vary significantly from module to module. We found that basic scanning functions performed by MPP are highly efficient, with overall performance significantly impacted by certain modules.

The anti-virus modules tend to consume more resources, with Pattern Authority being the most resource intensive. This is to be expected, since Pattern Authority straddles the roles of anti-virus, and anti-spam, unlike other modules.

The performance of SpamAssassin varies greatly, along with accuracy, based on the configuration of SpamAssassin. In our highly customized SpamAssassin installation, we have in excess of 20,000 custom rules, 6 individual RBL checks, and customized SpamCop URI checks on the body of each message. Even with all these checks, we see processing time of just a few seconds in the most drastic situations.

MPP also helped solved a problem we often encountered prior to deploying MPP regarding improperly formed e-mail messages. MPP parses each message while gathering the relevant information to apply the configuration to the scanning modules results. This allows MPP to prevent improperly formed messages from being handed off to the scanning modules. Which before MPP could cause enormous problems, resulting in a simple delay in processing, or in a worst case, crashing a scanner.

Managing Results

With MPP configured, deployed, and scanning messages, there are a number of choices for managing results. As mentioned earlier, the flagged messages can be quarantined on the mail server, rejected, deleted, or flagged for routing further down stream.

For messages infected with a virus, you will most likely want them quarantined on the server, or deleted completely. But for messages flagged as spam, we chose to flag them for processing down stream, either by the client, or the receiving mail server. Managing the spam quarantine in the current version of MPP is highly useful and effective, but in our environment, did not fit the needs of our clients.

The first issue we encountered with the current version of MPP is that SpamAssassin results are not injected into messages. This can make reviewing, and improving SpamAssassin rules more difficult in that you must review mail logs for the results. RAE has promised this issue will be resolved in an update in the near future.

Another minor issue results from the lack of a centralized configuration option, or management of quarantined messages across multiple servers. With two main production servers, and two backup servers, while still more efficient than our previous homegrown system, it still leaves us with four server configurations, and quarantine areas to manage.

RAE is working on a solution to part of this issue that we have tested. This summer, version 2.0 will introduce a quarantine area stored in a MySQL database. We hope that RAE considers storing configuration information in a MySQL database as well, which would help in managing multiple MPP installations across an enterprise.

Support

RAE Internet is a growing company, and in our experience has exhibited both enthusiastic support, as well as their fair share of growing pains. Overall, RAE's support for Mac OS X has been very good, unlike a fair number of far larger companies. RAE's primary support method is e-mail, though it is not difficult to get the refreshingly helpful, and honest staff on the phone if an issue calls for it.

One area that is sorely lacking, and an obvious result of their rapid growth, is the minimal documentation included with MPP, which is currently limited to the Read Me file, comments in the configuration file, and an online FAQ. RAE needs to focus on detailed technical documentation for MPP. As it stands now, learning the more advanced configuration options requires an exchange on the mailing list, or with technical support.

As an early adopter of MPP, in addition to receiving great support, it's been a pleasant surprise to see how responsive RAE has been to feature requests, and how rapidly those requests were made available to customers.

Is MPP worth it?

MPP is a rapidly maturing product that offers a good answer to a fundamental question an administrator has to ask. Do you commit to utilizing a single product to manage the majority of your email filtering, or do you spend the time, and effort piecing together a solution on your own?

The cost of MPP is determined based on which commercial scanning modules you choose, as well as the number of users. Since MPP is the management platform for the scanning solutions, the majority of licensing fees go towards the various commercial scanning modules they offer. However, MPP can be purchased to integrate with existing licenses for supported scanning products, like Sophos or Pattern Authority.

While Mac OS X is our platform of choice, MPP's license is not restricted by platform. If a company "outgrows" an existing platform, or requires larger servers, the investment in MPP is protected. The only restriction is that MPP is only supported on Unix operating systems.

With Mac OS X gaining traction in the enterprise market, MPP offers a significant value for those looking for an easily installed and managed solution for deploying anti-virus and anti-spam filtering.


Lon Baker is the founder of Speedymac LLC, a developer of e-mail services, and web applications. Speedymac has been providing e-mail centric solutions to businesses of all sizes for over 5 years. Lon has been succumbing to his Mac addiction for over 15 years.

 

Community Search:
MacTech Search:

Software Updates via MacUpdate

Skim 1.4.32 - PDF reader and note-taker...
Skim is a PDF reader and note-taker for OS X. It is designed to help you read and annotate scientific papers in PDF, but is also great for viewing any PDF file. Skim includes many features and has a... Read more
ForkLift 3.1.1 - Powerful file manager:...
ForkLift is a powerful file manager and ferociously fast FTP client clothed in a clean and versatile UI that offers the combination of absolute simplicity and raw power expected from a well-executed... Read more
Direct Mail 5.2.1 - Create and send grea...
Direct Mail is an easy-to-use, fully-featured email marketing app purpose-built for macOS. Create, send, and track great looking email campaigns that get results. Start your newsletter by selecting... Read more
Direct Mail 5.2.1 - Create and send grea...
Direct Mail is an easy-to-use, fully-featured email marketing app purpose-built for macOS. Create, send, and track great looking email campaigns that get results. Start your newsletter by selecting... Read more
Skim 1.4.32 - PDF reader and note-taker...
Skim is a PDF reader and note-taker for OS X. It is designed to help you read and annotate scientific papers in PDF, but is also great for viewing any PDF file. Skim includes many features and has a... Read more
ForkLift 3.1.1 - Powerful file manager:...
ForkLift is a powerful file manager and ferociously fast FTP client clothed in a clean and versatile UI that offers the combination of absolute simplicity and raw power expected from a well-executed... Read more
MarsEdit 4.0.5 - Quick and convenient bl...
MarsEdit is a blog editor for OS X that makes editing your blog like writing email, with spell-checking, drafts, multiple windows, and even AppleScript support. It works with with most blog services... Read more
MarsEdit 4.0.5 - Quick and convenient bl...
MarsEdit is a blog editor for OS X that makes editing your blog like writing email, with spell-checking, drafts, multiple windows, and even AppleScript support. It works with with most blog services... Read more
ffWorks 1.0.1 - Convert multimedia files...
ffWorks (was iFFmpeg), focused on simplicity, brings a fresh approach to the use of FFmpeg, allowing you to create ultra-high-quality movies without the need to write a single line of code on the... Read more
Civilization VI 1.0.7 - Next iteration o...
Sid Meier’s Civilization VI is the next entry in the popular Civilization franchise. Originally created by legendary game designer Sid Meier, Civilization is a strategy game in which you attempt to... Read more

Latest Forum Discussions

See All

Why doesn't mobile gaming have its...
The Overwatch League is a pretty big deal. It's an attempt to really push eSports into the mainstream, by turning them into, well, regular sports. But slightly less sweaty. It's a lavish affair with teams from all around the world, and more... | Read more »
Give Webzen’s new billiard game PoolTime...
Best known for producing hugely popular MMO titles, South Korean publisher Webzen is now taking aim at a different genre altogether. PoolTime is a realistic eight ball pool simulator, allowing you to compete in real-time matches against players... | Read more »
Let Them Come Guide - How to survive aga...
Let Them Come is all about making it as far as possible against overwhelming odds. Check out some of these tips to help you last a little longer in your unwinnable fight: [Read more] | Read more »
All the best games on sale for iPhone an...
Happy last day of the week. I hope you've been having a good one. I have. I saw ten doggos today. So because I'm in a good mood, I thought I'd round up all of the best games that are currently on sale on the App Store. [Read more] | Read more »
The very best games that came out for iP...
We're getting to the end of the first real, full, proper week of 2018. And in that time we've seen some pretty awesome games landing on the App Store. Of course, we've seen some absolute duffers as well. The sort of games that you look at and... | Read more »
Rusty Lake Paradise (Games)
Rusty Lake Paradise 1.4 Device: iOS Universal Category: Games Price: $2.99, Version: 1.4 (iTunes) Description: Jakob, the oldest son of the Eilander family, is returning to Paradise island after his mother passed away. Since her... | Read more »
Antihero Guide - Sneaky tricks to get ah...
Games of Antihero start out small and streamlined, but they quickly turn into long strategic conquests as you fight for control of the Victorian-era streets. If you find yourself struggling in the skullduggery department, here are a few things you... | Read more »
Here's why Niantic pulling Pokemon...
If there's one thing that Pokemon GO did well, it was bringing people together. I still remember seeing groups of people around the marina near where I live in the weeks after the game came out, all of them trying to grab some water Pokemon. There... | Read more »
Let Them Come (Games)
Let Them Come 1.0 Device: iOS Universal Category: Games Price: $1.99, Version: 1.0 (iTunes) Description: | Read more »
Why the iPhone X isn't the best bet...
It's always tempting to have the best thing. As game players we're almost conditioned to chase the biggest processor, the most power, the next generation. Even mobile gaming isn't immune to this push to progress - there are games on the App Store... | Read more »

Price Scanner via MacPrices.net

Amazon offers Silver 13″ Apple MacBook Pros f...
Amazon has new Silver 2017 13″ #Apple #MacBook Pros on sale today for up to $150 off MSRP, each including free shipping: – 13″ 2.3GHz/128GB Silver MacBook Pro (MPXR2LL/A): $1199.99 $100 off MSRP – 13... Read more
Sale: 12″ 1.3GHz MacBooks on sale for $1499,...
B&H Photo has Space Gray and Rose Gold 12″ 1.3GHz #Apple MacBooks on sale for $100 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 12″ 1.3GHz Space... Read more
Apple offers Certified Refurbished 2017 iMacs...
Apple has a full line of Certified Refurbished iMacs available for up to $350 off original MSRP. Apple’s one-year warranty is standard, and shipping is free. The following models are available: – 27... Read more
13″ MacBook Airs on sale for $120-$100 off MS...
B&H Photo has 2017 13″ 128GB MacBook Airs on sale for $120 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13″ 1.8GHz/128GB MacBook Air (MQD32LL/A): $... Read more
15″ Touch Bar MacBook Pros on sale for up to...
Adorama has Space Gray 15″ MacBook Pros on sale for $200 off MSRP. Shipping is free, and Adorama charges sales tax in NJ and NY only: – 15″ 2.8GHz MacBook Pro Space Gray (MPTR2LL/A): $2199, $200 off... Read more
21″ 3.4GHz 4K iMac on sale for $1399, $100 of...
Adorama has the 21″ 3.4GHz 4K #Apple #iMac on sale today for $1399. Their price is $100 off MSRP. Shipping is free, and Adorama charges sales tax in NJ and NY only: – 21″ 3.4GHz 4K iMac (MNE02LL/A... Read more
B&H offering 13″ Apple MacBook Pros for u...
B&H Photo has 13″ MacBook Pros on sale for up to $75-$120 off MSRP. Shipping is free, and B&H charges sales tax for NY & NJ residents only: – 13-inch 2.3GHz/128GB Space Gray MacBook Pro (... Read more
B&H continues to offer clearance 2016 15″...
B&H Photo has clearance 2016 15″ #MacBook Pros available for up to $800 off original MSRP. Shipping is free, and B&H charges NY & NJ sales tax only: – 15″ 2.7GHz Touch Bar MacBook Pro... Read more
The cheapest 15″ Apple MacBook Pro available...
B&H Photo has the 15″ 2.2GHz MacBook Pro available for $200 off MSRP including free shipping plus NY & NJ sales tax only: – 15″ 2.2GHz MacBook Pro (MJLQ2LL/A): $1799 $200 off MSRP Apple has... Read more
Beats sale continues: $30-$80 off headphones,...
Walmart has Beats by Dr. Dre on sale on their online store for $30-$80 off MSRP (#Beats #Walmart), depending on the item: – Powerbeats3 Wireless Earphones: $134, save $65 – BeatsX Earphones: $109,... Read more

Jobs Board

*Apple* Retail - Multiple Positions - Apple,...
Job Description:SalesSpecialist - Retail Customer Service and SalesTransform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
Site Reliability Engineer, *Apple* Pay - Ap...
# Site Reliability Engineer, Apple Pay Job Number: 113356036 Santa Clara Valley, California, United States Posted: 12-Jan-2018 Weekly Hours: 40.00 **Job Summary** Read more
UI Tools and Automation Engineer, *Apple* M...
# UI Tools and Automation Engineer, Apple Media Products Job Number: 86351939 Santa Clara Valley, California, United States Posted: 11-Jan-2018 Weekly Hours: 40.00 Read more
*Apple* Retail - Multiple Positions - Apple,...
Job Description: Sales Specialist - Retail Customer Service and Sales Transform Apple Store visitors into loyal Apple customers. When customers enter the store, Read more
UI Tools and Automation Engineer, *Apple* M...
# UI Tools and Automation Engineer, Apple Media Products Job Number: 113136387 Santa Clara Valley, California, United States Posted: 11-Jan-2018 Weekly Hours: 40.00 Read more
All contents are Copyright 1984-2011 by Xplain Corporation. All rights reserved. Theme designed by Icreon.