System-wide virus and spam scanning

Installing qmail-scanner, Clam Antivirus and SpamAssassin under FreeBSD

$Date: 2002/10/25 09:48:44 $

This document describes how to effect system-wide virus and spam scanning of incoming email. The approach is based on the qmail mail transport agent, and is not applicable to sites running sendmail. This document describes how to install qmail-scanner, an enhancement for qmail that allows incoming mail to be passed through third-party filters prior to normal local delivery. The two filters described in this document are Clam AntiVirus, an open source virus scanning package, and SpamAssassin, an open source spam detector.


Table of Contents
1. Pre-requisites
2. Installing Clam AntiVirus
3. Installing SpamAssassin
4. Installing qmail-scanner
A. Fetching and installing Perl modules from CPAN
B. Contacting the authors

1. Pre-requisites

The following instructions are intended to be comprehensive, but there are at least these pre-requisites:

It is necessary to install Clam AntiVirus and SpamAssassin prior to installing qmail-scanner, as the latter tries to automatically detect available third-party scanners at installation time. There are no dependencies between Clam AntiVirus and SpamAssassin in the following approach — installation of either can be omitted if that functionality is not required.


2. Installing Clam AntiVirus

2.1. Building and installing from source

There is no FreeBSD Port for Clam AntiVirus, so it needs to be built by hand from the source. Firstly, obtain and unpack the source:

Editor's note: this is no longer true. You can find a port of the stable release in /usr/ports/security/clamav and a port of the development release in /usr/ports/security/clamav-devel

# cd /usr/local
# mkdir clamav
# cd clamav
# fetch http://clamav.elektrapro.com/stable/clamav-0.51.tar.gz
# gunzip clamav-0.51.tar.gz
# tar -xvf clamav-0.51.tar.gz

Create new user and group names for Clam AntiVirus:

# pw groupadd clamav
# pw useradd clamav -g clamav -d /nonexistent -c "Clam Antivirus" \
  -s /sbin/nologin

Now configure and install Clam AntiVirus:

# cd clamav-0.51
# ./configure
# make install

2.2. Testing the installation

You should now read the documentation for clamscan (man clamscan, or read the online documentation). You can test the scanner by running:

# clamscan --recursive --log=/tmp/clamscan.log /usr/home

Obviously this can be run on the base directory of your choice, and the log file location is also arbitrary. Next, use the freshclam command to update your databases:

# freshclam --verbose

2.3. Running freshclam as a daemon

The freshclam database updater can be run as a daemon. Firstly, create a log file for freshclam:

# touch /var/log/freshclam.log
# chmod 644 /var/log/freshclam.log
# chown clamav:clamav /var/log/freshclam.log

Create following script as /usr/local/etc/rc.d/clamav.sh to start up freshclam as a daemon at boot time, and cause it to die gracefully at shutdown:

#!/bin/sh
#
# Startup / shutdown script for Clam Antivirus

case "$1" in
    start)
        /usr/local/bin/freshclam -d -c 2 -l /var/log/freshclam.log
        echo -n ' freshclam'
        ;;

    stop)
        /usr/bin/killall freshclam > /dev/null 2>&1 \
        && echo -n ' freshclam'
        ;;

    *)
        echo ""
        echo "Usage: `basename $0` { start | stop }"
        echo ""
        exit 64
        ;;
esac

Ensure that the script is executable:

# chmod 744 /usr/local/etc/rc.d/clamav.sh

2.4. Running clamscan on a regular basis

If you have a filesystem directory tree that you think would benefit from regular virus scanning (anything accessible from any Microsoft Windows machines on your LAN would probably be candidates), you can set up a cron job to run clamscan on a regular basis. Read the Clam AntiVirus documentation and decide which options to clamscan are appropriate for your site. For example, you may not wish to specify the --remove option, and you may wish to --exclude any number of files or directories from scanning. As an example, the following entry appended to /etc/crontab will scan /usr daily at 6.00am:

0   6   *   *   *   root   /usr/local/bin/clamscan --recursive
                           --infected
                           --exclude /usr/local/share/clamav/viruses.db
                           --exclude /usr/local/share/clamav/viruses.db2
                           --log=/var/log/clamscan.log
                           /usr/home
Note

The line in /etc/crontab is shown broken here to improve readability, but should appear as a single line in the file.


3. Installing SpamAssassin

3.1. Building and installing from source

There is a FreeBSD Port for SpamAssassin, though it will build an old version (2.41). Either build that port with:

# cd /usr/ports/mail/p5-Mail-SpamAssassin
# make
# make install

or build the latest version by hand from the source as follows. Firstly, obtain and unpack the source:

# cd /usr/local
# mkdir spamassassin
# cd spamassassin
# fetch http://spamassassin.taint.org/released/Mail-SpamAssassin-2.43.tar.gz
# gunzip Mail-SpamAssassin-2.43.tar.gz
# tar -xvf Mail-SpamAssassin-2.43.tar
# cd Mail-SpamAssassin-2.43

Now, build the Makefile with perl:

# perl Makefile.PL
Note

At this point, perl will warn you of any dependencies on Perl packages your system is missing. Fetching and installing Perl packages is described in an Appendix below. Fetch and install any packages required before proceeding.

Now build SpamAssassin:

# make
# make install

3.2. Testing the installation

You should now test SpamAssassin on the sample files provided. Firstly, test some known spam:

# spamassassin -t < sample-spam.txt > spam.out

View the resulting file, spam.out. SpamAssassin should add the following headers to the message:


X-Spam-Status: Yes, hits=14.7 required=5.0
        tests=ALL_CAPS_HEADER,CALL_FREE,DATE_IN_PAST_24_48,
              DRASTIC_REDUCED,FROM_HAS_MIXED_NUMS,HOME_EMPLOYMENT,
              INVALID_DATE,INVALID_MSGID,LINES_OF_YELLING,
              MSGID_HAS_NO_AT,NO_REAL_NAME,ONCE_IN_LIFETIME,REMOVE_SUBJ,
              SMTPD_IN_RCVD,SPAM_PHRASE_21_34,UNDISC_RECIPS
        version=2.43
X-Spam-Flag: YES
X-Spam-Level: **************
X-Spam-Checker-Version: SpamAssassin 2.43 (1.115.2.20-2002-10-15-exp)

Additionally, there will be a banner explaining in detail what tests were failed.

Next, test SpamAssassin with a piece of genuine email:

# spamassassin -t < sample-nonspam.txt > nonspam.out

This should add only the following headers to the mail, indiciating the message is not considered spam:

X-Spam-Status: No, hits=0.9 required=5.0
        tests=GAPPY_TEXT,LINES_OF_YELLING,PGP_SIGNATURE,
              SPAM_PHRASE_02_03,TO_BE_REMOVED_REPLY
        version=2.43
X-Spam-Level:
Note

SpamAssassin's only action is to mark mail that it considers spam with the X-Spam- headers. It does not delete or even remove spam. Another agent is required in the chain to move the spam once detected. Instructions are given below for a simple per-user procmail recipe for moving spam to a separate folder.


3.3. Running SpamAssassin as a daemon: spamd

If SpamAssassin was installed from the Ports System, a startup script for spamd will have been placed in /usr/local/etc/rc.d/spamd.sh. If SpamAssassin was installed by hand, you should now create a startup script for spamd (the daemon version of SpamAssassin), similar to the one created above for freshclam. Create the following script as /usr/local/etc/rc.d/spamd.sh:

#!/bin/sh
#
# Startup / shutdown script for SpamAssassin daemon

case "$1" in
    start)
        /usr/bin/spamd -a -d && echo -n ' spamd'
        ;;

    stop)
        spamdpid=`ps -ax | grep spamd | grep -v grep | grep -v sh | awk '{ print $1 }'`
        if [ "$spamdpid" != "" ]; then
                kill $spamdpid > /dev/null 2>&1
                echo -n " spamd"
        fi
        ;;

    *)
        echo "Usage: `basename $0` {start|stop}" >&2
        ;;
esac

exit 0

Remember to make the script executable, then use it to start spamdqmail-scanner will only find the SpamAssassin daemon if it is running at the time of install:

# chmod 744 /usr/local/etc/rc.d/spamd.sh
# /usr/local/etc/rc.d/spamd.sh start

3.4. Using procmail to filter the spam

As noted above, SpamAssassin only tags spam with X-Spam- headers. If you don't do anything else, you'll still receive spam in your mailbox — it will just be identified as spam by those headers. One common solution is to use procmail as a mail delivery agent and instruct it to place the spam in a Maildir of its own. There is a lot of good documentation on installing and running procmail, and a thorough treatment of that complex program is beyond the scope of this document. If you have procmail installed at your site already, though, or even if you are prepared to install it from the Ports System just to redirect SpamAssassin-tagged spam, the following is a minimal procmail recipe to redirect spam to the Maildir $HOME/Maildir/.Spam/:

:0:
* ^X-Spam-Status: Yes
$HOME/Maildir/.Spam/

This recipe would be placed in each user's .procmailrc file. In addition, placing it in the file /usr/share/skel/dot.procmailrc will ensure that any newly created users will have a .procmailrc file generated in their home directory. Each user will also need to have a .Spam Maildir created for them. For example, to create the Maildir for paulh, enter:

# su paulh
# cd $HOME
# /var/qmail/bin/maildirmake Maildir/.Spam
# exit

In order to get qmail to call procmail, each user's .qmail file should contain the following:

|IFS=' ' && exec /usr/local/bin/procmail -f- || exit 75

Again, to ensure all new users have this .qmail created for them, replace the contents of /usr/share/skel/dot.qmail with the line above.

Note

Installing and running procmail is non-trivial. Read the documentation and the numerous FAQs and How-Tos if you plan to do this.


4. Installing qmail-scanner

4.1. Installing maildrop

qmail-scanner depends on the reformime command, available as part of the maildrop package. maildrop is available in the FreeBSD Ports System, and is easily installed:

# cd /usr/ports/mail/maildrop
# make
# make install

4.2. Installing and building from source

There is no FreeBSD Port for qmail-scanner, so it needs to be built by hand from the source. Firstly, obtain and unpack the source:

# cd /usr/local
# mkdir qmail-scanner
# fetch http://telia.dl.sourceforge.net/sourceforge/qmail-scanner/qmail-scanner-1.14.tgz
# gunzip qmail-scanner-1.14.tgz
# tar -xvf qmail-scanner-1.14.tar
# cd qmail-scanner-1.14

Now run the configure script in help mode to view the options:

# ./configure --help

You may wish to explicitly specify some of the options for which the defaults are unsuitable at your site. For example, you may wish to change --admin and --notify at a minimum. The run the configure script using your options and the --install option. For example:

# ./configure --admin postmaster --notify "recips,admin" --install

You need to set the SUID bit on the suidperl command:

# chmod 4511 /usr/bin/suidperl
Note

This may be a security risk. You need to evaluate this action based on the security policy at your site. qmail-scanner will not operate without this change, however.

qmail-smtpd needs to be instructed to use the qmail-scanner-queue.pl script in /var/qmail/bin instead of the standard qmail-queue binary. If your site uses tcpserver to handle connections to qmail-smtpd (as described in Installing qmail under FreeBSD), /etc/tcp.smtp should be updated to set the QMAILQUEUE environment variable. The precise contents of this file will vary between sites depending on you LAN configuration. The /etc/tcp.smtp file constructed in Installing qmail under FreeBSD would be modified as follows:

192.168.0.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"
127.:allow,RELAYCLIENT="",QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"
:allow,QMAILQUEUE="/var/qmail/bin/qmail-scanner-queue.pl"

Now rebuild the ruleset for tcpserver:

# /usr/local/bin/tcprules /etc/tcp.smtp.cdb /etc/tcp.smtp.tmp < /etc/tcp.smtp

Finally, stop and restart the qmail binaries:

# /usr/local/etc/rc.d/qmail.sh stop
# /usr/local/etc/rc.d/qmail.sh start

4.3. Testing the installation

The qmail-scanner distribution provides a script and some test files containing virus signatures to test the virus scanner. Run these through qmail-scanner now:

# cd /usr/local/qmail-scanner/qmail-scanner-1.14
# ./contrib/test_installation.sh -doit

This will send three emails to the address you specified as --admin during the qmail-scanner installation. The first should arrive unmodified (though it will have an X-Spam-Status: header if you have set up SpamAssassin), but the second and third should be caught by either the internal (simple) virus scanner of qmail-scanner or by Clam AntiVirus. Email caught by qmail-scanner is deposited in /var/spool/qmailscan/quarantine in Maildir format.


A. Fetching and installing Perl modules from CPAN

At several points during the installation of the Perl-based applications, perl may complain about dependencies on packages not present on your system. This is easily solved by installing the packages it requires from CPAN. The easiest way is to use a CPAN shell:

# perl -MCPAN -e shell

If you are running the CPAN module for the first time, there may be some setup required — just follow the prompts. When you get the cpan> prompt, install the packages required. For example, to install Time::HiRes, simply enter:

cpan> install Time::HiRes

You can instruct the shell to install as many packages as you need in this manner.


B. Contacting the authors

This document was written by Paul Hoadley and Eric Parsonage. This document describes what we did to get qmail-scanner co-operating with Clam AntiVirus and SpamAssassin on two FreeBSD 4.7 systems. Your mileage may vary. If you notice any errors in this document, or your experience with the software used was vastly different, please let us know.