This document addresses Netscape Messaging Server only. If you are looking for UBE filters for Netscape Communicator or Navigator, you are at the wrong page.
If you are using NMS version 4.0 and above, you should upgrade to version 4.15 and use the Anti-Relay plug-in instead of UBE filters to stop unauthorized relaying. Click here for the Netscape Anti-Relay documentation and definitely look here for more details.
Many administrators have problems properly configuring the UBE filters to block unauthorized relaying, primarily due to the poor quality of the Netscape documentation (3.5, 4.1, and knowledge base), not fully understanding RFC 821 and RFC 822, and inexperience using Extended Regular Expressions (EREs). (Netscape Messaging Server documentation can be found here).
If you aren't sure about the relay status of your mail server, or you want to verify that your configuration is working properly, you can check your server for relaying by using the MAPS Transport Security Initiative, or Sam Spade. These tests may report that your server may be vulnerable to relaying even if you have the UBE filters properly configured. This is because NMS accepts a message before the UBE filters are applied. You should check your NMS log files to verify if the message was actually relayed.
The following filter.cfg file has been successfully used in many installations to both block unauthorized relaying and perform basic UBE filtering. This file is generic enough to be used for any installation after simple modifications which define your enviroment are made.Do not use the Administrator web interface to input or edit the filter.cfg file, or to commit changes to it! This filter.cfg file contains strings which confuse the Admin server pages. Clearly 90% of the email that I receive about this web page is because that person used the Admin web interface anyway and they don't understand why their system fails the ORBS tests. Usually the '%' character is missing from the ":ChkAddr" line. In addition, if you use the Admin web pages to turn the UBE filters off and then back on, you may need to edit or restore your filter.cfg file. Please maintain a backup copy.
Under NMS version 4.0 and higher, the default UBE filter file name is 'UBEfilter.cfg', not 'filter.cfg'.
This filter.cfg file contains the following enviroment dependencies which must be changed before you use this file: The local network is 192.10.20, the domain names handled by this server are 'mydomain.com' and 'otherdom.com'. In addition, a user named 'spam' has been created to which all messages identified as UBE will be sent. This user's mailbox should be checked daily for false positives.
In order to enable UBE filters and use this file, you must have Plugin Configuration set to On under the Plugins tab of the Admin server page for your NMS configuration, and "parseheader:1" must be set in the filter.opt file from the "Edit Option File" button at the bottom of the UBE filters page.If you have problems with cut and paste and don't want to type the file in by hand, right click here, select "Save Link As", and save as text format if that is an option.
----------------------filter.cfg (do not incclude this line)--------------------
# filter.cfg for NMS 3.5 and above. 28 April 1999.
# Author: Bob Poortinga
#
# Do not use the Administrator web interface to input or edit this file,
# or to commit changes! Use a text editor such as Wordpad or vi.
# Do not use Notepad. It may create a file with an incomplete end of line.
# I will not answer any email if you use the Admin interface!
#
# First check if user is authenticated. If so, no further checking is
# necessary & exit. This header appears ONLY if the user's password has
# been successfully verified on this server using Authenticated SMTP.
#
# If you are using multiple NMSs that exchange mail and use AUTH SMTP, you
# may need to remove or modify this line.
#
Auth-Sender:envonly ".+" EXIT
#
# Now check if sending system is on local network (127.0.0.1 is ALWAYS needed)
# These patterns have been specifically designed to reject forgeries
# perpetrated by bogus rDNS entries.
#
Host-From:envonly "\[127\.0\.0\.1\][^[]*$" EXIT
#
# Change the following to match your network. Change only the network numbers.
# For each additional trusted network, add a new line. For class B networks
# use a pattern of the form "\[128\.10\.[0-9]+\.[0-9]+\][^[]*$".
#
# If your server is behind a firewall, you may have to use a different
# strategy depending on your firewall and network configuration.
# In some configurations, you can assume that any message with a "Host-From"
# that matches the firewall IP address originates from a remote system.
# For this case, add this line (assuming the firewall IP is 192.10.20.1)
#
# Host-From:envonly "\[192\.10\.20\.1\][^[]*$" JUMP "ChkAddr"
#
Host-From:envonly "\[192\.10\.20\.[0-9]+\][^[]*$" EXIT
#
# If we get here, mail is coming from foreign system
# Check for relay attempt in SMTP addressing
# Updated 28 May 1999 to handle path hacks ("!" in address)
# Updated 24 July 1999 to fix logic flaw for multiple Channel-To's.
# Also added additional pattern to check for <"user@x.com"@y.com>
# Prior version had '!JUMP' which would have accepted any message
# with a least one good address. Thanks to Paul Pinocci of Booz,
# Allen & Hamilton for calling this to my attention.
#
:ChkAddr Channel-To:envonly "<@|<.*[%,:!]|<.*@.*@" JUMP "Bounce"
#
# Check all recipients against our primary local domain names. If not a
# match, then the message is a relay attempt and we will bounce (REJECT) it.
# Modify this line with your domains. Do NOT use multiple Channel-To
# filters to match your local domains. Doing so results in opening up
# your server to relaying. If you have a single domain, use a pattern
# of the form "[.@]ourdomain\.com>". If there are too many domains to
# fit on one line, you will have to write an external program or script
# to verify the recipient domain which can be called with the RUN action.
# (See note below on using the RUN action).
#
# These patterns assume that you are using .COM domains. If your domain
# is based on another Top Level Domain (TLD), you need to change them
# accordingly (these examples are patterns to be used in the :ChkRcpt line
# below, do *not* uncomment or edit the examples!) e.g.
# "[.@](mydomain|otherdom)\.org>" or
# "[.@](mydomain|otherdom)\.fr>" or
# "[.@](mydomain|otherdom)\.co\.uk>" or
# "[.@](mydomain|otherdom)\.k12\.portland\.me\.us>
#
# For a single domain, use:
# "[.@]mydomain\.com>"
#
# If you are using two different TLDs, you will have to use a pattern of
# the form (assuming .NET and .CO.UK TLDs):
# "[.@]((mydomain|otherdom)\.net|(name3|name4)\.co\.uk)>"
#
# The trailing ">" at the end of the pattern is required to guarantee the
# proper match. Do not remove it.
#
:ChkRcpt Channel-To:envonly "[.@](mydomain|otherdom)\.com>" !JUMP "Bounce"
Host-From:envonly ".*" JUMP "RcptOk"
#
# Someone is trying to relay. Bounce the message. If return address is
# invalid, messsage will end up in our postmaster mailbox. Another option
# would be to send (DROP) it to designated local mailbox (such as "relay")
# like this ":Bounce Host-From:envonly ".*" DROP "relay"
#
:Bounce Host-From:envonly ".*" REJECT "Non-local addressee. We do not relay!"
Host-From:envonly ".*" EXIT
#
# When we get here, message is destined for local mailbox.
# Check for common spam fingerprints. If found, re-route (DROP) message to
# user "spam"'s mailbox. Check daily for false positives. Some of these
# patterns may seem somewhat cryptic, but are based on analysis of thousands
# of UBEs and should trigger few false positives.
#
:RcptOk Received "GAA.*-0600.*EST" JUMP "Spam"
Received "XAA.*-0700.*EDT" JUMP "Spam"
Received "xxxxxxxxxxxxxxxxxxxxx" JUMP "Spam"
Received "untrace?able" JUMP "Spam"
Received "from (baby|bewellnet|kllklk) " JUMP "Spam"
To "Friend@public\.com" JUMP "Spam"
To "user@the[-_]internet" JUMP "Spam"
Date "/[0-9]+/.+[AP]M.+Time" JUMP "Spam"
Subject "^\(?ADV?[:;)]" JUMP "Spam"
Message-ID "<>" JUMP "Spam"
Message-Id "<>" JUMP "Spam"
Message-Id "<(419\.43|989\.28)" JUMP "Spam"
X-MimeOLE "MimeOLE V[^0-9]" JUMP "Spam"
#
# Added 20-Jun-1999. Appears to be broken spamware.
#
MIME-Version "1.0From" JUMP "Spam"
#
# Added 28-July-1999. Check X-Mailer for spamware.
#
X-Mailer "DiffondiCool" JUMP "Spam"
X-Mailer "Emailer Platinum" JUMP "Spam"
X-Mailer "eMerge" JUMP "Spam"
X-Mailer "Crescent Internet Tool" JUMP "Spam"
#
# Added 4-Apr-2000. Check X-Mailer for Cybercreek Avalanche
#
X-Mailer "Avalanche" JUMP "Spam"
#
# Added 28-July-1999. Bcc to 10 or more recipients
#
Bcc "@.*@.*@.*@.*@.*@.*@.*@.*@.*@" JUMP "Spam"
#
# Added 21-Oct-1999. Subject contains 20 or more consecutive spaces
#
Subject " " JUMP "Spam"
#
# Added 31-Mar-2000. Invalid headers from MyGuestBook.exe CGI spamware
#
MessageID "<.+>" JUMP "Spam"
X-References "0[A-Z0-9]+, 0[A-Z0-9]+$" JUMP "Spam"
X-Other-References "0[A-Z0-9]+$" JUMP "Spam"
X-See-Also "0[A-Z0-9]+$" JUMP "Spam"
#
# Updated 28-Apr-1999. Check for "Sender", "Resent-From", or "Resent-By"
# before "X-UIDL". If found, then exit.
#
Sender ".+" EXIT
Resent-From ".+" EXIT
Resent-By ".+" EXIT
#
# Updated 19-May-1999. Check for "X-Mozilla-Status" before "X-UIDL".
#
X-Mozilla-Status ".+" EXIT
#
# Updated 20-Jul-1999. Check for "X-Mailer: Internet Mail Service"
# before "X-UIDL".
#
X-Mailer "Internet Mail Service" EXIT
#
# Updated 25-Oct-1999. Check for "X-ID" before "X-UIDL".
#
X-ID ".+" EXIT
#
# X-UIDL is a POP3 header that should normally not be seen
#
X-UIDL ".*" JUMP "Spam"
#
# Some headers are valid only for the Pegasus Mail client. So first check
# for Pegasus header and exit if found. If not found, check for
# invalid headers: "Comments: Authenticated sender", "X-PMFLAGS" and "X-pmrqc".
#
X-mailer "Pegasus" EXIT
#
# Added 27-Aug-1999. Pegasus now uses X-Mailer instead of X-mailer.
#
X-Mailer "Pegasus" EXIT
#
# Added 25-Oct-1999. Check for X-Confirm-Reading-To.
#
X-Confirm-Reading-To ".+" EXIT
#
# Check for invalid Pegasus headers
#
Comments "Authenticated sender" JUMP "Spam"
X-PMFLAGS ".*" JUMP "Spam"
X-Pmflags ".*" JUMP "Spam"
X-pmrqc ".*" JUMP "Spam"
Host-From:envonly ".*" EXIT
#
# Spam detected. Drop all recipients and send to "spam" mailbox. Check this
# mailbox daily for false positives.
#
:Spam Host-From:envonly ".*" DROP "spam"
Host-From:envonly ".*" EXIT
------------------end of filter.cfg (do not include this line)-----------------
Eddy Beliveau has put together an excellent resource site for NMS
called Eddy's NMS toolbox.
Mark Grifman has a very nice site including a NMS discussion group at
Grifman's Home.
John Navarro has a collection of Perl scripts and other tips at
John's Goody Bag
All Netscape technical articles about Messaging Server 3.x can be found here. Beware: some of the articles are just plain wrong. Your mileage may vary. As my buddie, Dave, always admonishes me: "One test is worth a thousand expert opinions."
*Important update* - The RUN action is no longer supported in
NMS version 4.1.
Click
here for details.
The "RUN" action can be used to extend the capabilities of the UBE filter in many ways. Unfortunately, the Netscape documentation does not do a very good job of explaining how to use it, and is plain wrong in some areas. My experience and testing has taught me a great deal in using the "RUN" action in Unix environments which may be of interest to you. If you are using NT, some of this information may be applicable. Since I haven't tried any of this on NT, I can make no guarantees. Unfortunately, at this point, you are on your own. Here is a Netscape knowledge base article about executing batch files in NT with the RUN action.
To execute an external program from the UBE filter, you call it with the "RUN" action. The file must exist in your postoffice (spool) directory for 3.x (instanceDirectory/smtp-bin/plugins/ for 4.x)and have the proper execute permissions granted to the user under which NMS runs. The only value that can be passed back to the UBE filter from an external program is the exit code. When a program (or script) is executed by the "RUN" action, it is passed two arguments: the full pathname of the RFC 822 message header file, and the full pathname of the RFC 822 message body file. STDIN is not connected to anything. This is contrary to the Netscape documentation which states "Once the program is started the UBE filter will pipe the header and body of the message so that you can do whatever you want to it." If you need to access other files from your program, you must use fully qualified path names for your files.
Since your program is given the names of the header and body files, it can read from and write to them. You can write a program that scans the body for patterns. (This is contrary to a Netscape technical article which states that it cannot be done short of writing a plugin.) You can also add your own custom headers to messages. Just remember that user defined headers should start with "X-". You can even replace the entire message body if you want. For instance, your program might detect that the message body is very large, e.g. > 10 MB. The external program could copy the message to an FTP directory and replace the message body with a note telling the user of it's name and location.One important point to consider in implementing an external program is that should it fail for some reason, the failure should not be interpreted as a valid normal termination. To put it another way, if you are checking the exit code of an external program in your UBE filters, the failure of your program should not result in all of your mail being bounced or otherwise being undeliverble. Suppose you are using a Perl script to check for spam. If it detects spam, the mail will be redirected to another mailbox. You might design the program such that if spam is detected, the exit code is 1, otherwise 0. Your UBE filter would match against a exit code to be zero or non-zero. If it is non-zero, it would be considered spam. Now consider that your system might run out of swap space or file handles, and your script fails to execute. A non-zero exit code is returned to your UBE filter, and all of your mail is then considered spam.
Since the external program is forked from the server daemon, it inherits the server's environment. There are some very useful environment variables available. Many of them are defined in the /etc/netscape.mail.conf file. One can then assume that this file simply defines a set of environment variables that are used to configure the server daemon. Some of more useful ones are:
ProgramDir=/opt/netscape/suitespot/bin/mail/server PostOffice=/var/spool/postoffice MailboxDir=/var/spool/mailboxSo the path of the control file directory would be "${PostOffice}/control" and the server log directory path is "${PostOffice}/log".
The following shell script, chkdomain.sh, can be called by the "RUN" action to check if all the recipients of a message are local to this server. It assumes that all of the local mail domain names are contained, one per line, in a file named /etc/mail.domains. This script will return an exit code of "1" if all recipients are local, and "0" if any of the recipients are remote.
--------------------------------chkdomain.shh------------------------------- #!/usr/bin/sh # # chkdomain.sh - Bob Poortinga, Technology Service Corp. 27-March-1999. # Updated 30-Aug-1999 for Solaris/Bourne shell. # # Verify mail recipients against /etc/mail.domains. # # This script is designed to be called from the NMS UBE filters with the # "RUN" action. The /etc/mail.domains file contains domain name PATTERNS, # one per line, of the form: domain1\.com$, k12\.bloomington\.in\.us$ or # somedomain\.co\.uk$. The dollar sign is necessary to mark the end of # each domain and the \. represents a literal period. # DOMAINFILE=/etc/mail.domains # # Set default path # PATH=/usr/bin:/bin:$PATH # Use for non-Solaris # PATH=/usr/xpg4/bin:/usr/ucb:/usr/bin:/bin:$PATH # Use for Solaris # # The first argument to this script is the path name of the message header # file. We want the path name of the message control file. We can get this # by changing the "-H" at the end of the message path to a "-C" and replacing # the "messages" part of the path with "control". # CTLFILE=`echo $1 | sed -e 's/-H$/-C/' -e 's,/messages/,/control/,'` # # Grep out the Channel-To lines: and use sed to extract only the domain name # The first sed command strips out everything up to and including the '@'. # The second sed command deletes the trailing '>'. # The last grep compares each recipient domain to see if there is a pattern # in the /etc/mail.domains file that matches and exits with a zero exit code # if a matching pattern is not found. # # If your version of grep does not support the -f option or you are having # other problems with grep, you may need to download and compile the latest # version of GNU grep from ftp://ftp.gnu.org/gnu/grep # grep '^Channel-To:' < $CTLFILE | sed -e 's/^.*\@//' -e 's/>.*$//' | grep -q -i -v -f $DOMAINFILE && exit 0 exit 1 -------------------------------end of chkdommain.sh------------------------------The following lines are to be used in place of the "Channel-To:" line that checks the recipient's domain name in the filter.cfg file above if you have more domains than can fit on one line.
Host-From:envonly ".*" RUN "chkdomain.sh" $& "0" JUMP "Bounce"
[More to follow. Please check back later.]
In response to the huge amount of email abuse that has been occuring, several blacklists have been developed in an effort to protect networks from this abuse. These blacklists have different purposes depending on how you want to protect your network. The most well known is probably the MAPS Realtime Blackhole List (RBL). Other blacklists include the MAPS Dial-Up List (DUL), the MAPS Relay Spam Stopper, and the Open Relay Behavior-modification System. This list is not all inclusive, and other blacklists exist for other purposes.
I strongly recommend that you throughly research each blacklist that you decide to use, or you may find much legitimate mail may be affected. In particular, if you are an ISP or allow users outside your network to access your SMTP server, you should be extremely cautious in using any of the dial-up lists, or you may find that your own users are unable to send mail.
If you wish to access any of these blacklists from the UBE filters, the following shell script, chkblacklists.sh, can be called with the "RUN" action and should work on any Unix system. I considered a Perl script, but decided that a shell script could be universally installed without the System Administrator having to download and install Perl.
chkblacklists.sh can be used in either client mode, or server mode. For each blacklist in which a sending mail host is found, a header of the form: 'X-Blacklist: {blacklist}' is added to the RFC 822 message headers on which the client mail readers can be configured to filter. For server mode, the script returns an exit code which can be checked in the UBE filters, and an appropriate action such as "REJECT" or "DROP" can be performed. The exitcode is "1" if the host is not blacklisted, or "0" if found on at least one blacklist.
For client mode simply add this line in your filter.cfg after you have checked for trusted hosts and networks:
Host-From:envonly ".*" RUN "chkblacklists.sh"
For server mode, add this additional line after the above line. Modify the action to suit your needs:
$& "0" JUMP "Spam"I recommend that you initially run only in client mode until you have assessed its impact on your mail system.
--------------------------------chkblacklistts.sh-------------------------------
#!/usr/bin/ksh
#
# chkblacklists.sh - Bob Poortinga, Technology Service Corp. 27-March-1999.
# Updated 30-Aug-1999 for Solaris/Bourne shell.
#
# Check if remote host if on a blacklist. This script will add a message
# header of the form "X-Blacklist: {blacklistdomain}" for each blacklist
# in which the remote host is found. This can be used as the basis for
# client filters. This script will return an exit code of 0 if the
# remote host is found in any of the blacklists, and 1 otherwise.
#
# This script is designed to be called from the NMS UBE filters with the
# "RUN" action.
#
# The /etc/mail.blacklists file contains domain names, one per line, to be
# used to access blacklists such as the MAPS RBL through nslookup.
#
BLACKLISTFILE=/etc/mail.blacklists
#
# Set default path (you may need to add the path for nslookup)
#
PATH=/usr/bin:/bin:$PATH
# PATH=/usr/xpg4/bin:/usr/ucb:/usr/bin:/bin:$PATH # For Solaris
#
# The first argument to this script is the path name of the message header
# file. We want the path name of the message control file. We can get this
# by changing the "-H" at the end of the message path with a "-C" and replacing
# the "messages" part of the path with "control".
#
HDRFILE=$1
CTLFILE=`echo $HDRFILE | sed -e 's/-H$/-C/' -e 's,/messages/,/control/,'`
#
# Next get the remote host IP address and split it up into octets so that
# we can convert it into IN-ADDR form for the lookups.
#
SAVEIFS=$IFS
IFS='.
'
grep '^Host-From:' < $CTLFILE |
sed -e 's/^.*\[//' -e 's/\][^[]*$//' |
read ip1 ip2 ip3 ip4
IFS=$SAVEIFS
ip_reverse=${ip4}.${ip3}.${ip2}.${ip1}.
#
# Now read each line in the /etc/mail.blacklist file that does not begin with
# a '#' and use it to perform the lookups.
#
exitcode=1
grep -v '^[:space:]*#' $BLACKLISTFILE |
while read blacklist; do
lookup=${ip_reverse}${blacklist}
nslookup $lookup 2>>/dev/null | grep -q '127\.0\.0\.[2-9]' &&
echo "X-Blacklist: $blacklist" >>$HDRFILE && exitcode=0
done
exit $exitcode
-----------------------------end of chkblackklists.sh----------------------------
This script reads a file /etc/mail.blacklists, which contains the domain names, one per line, which are used to access each blacklist. The blacklist file may also contain comments which have a '#' at the beginning of each line. Here is a sample /etc/mail.blacklists file that accesses the MAPS RBL and RRSS. Uncomment each blacklist domain that you decide to use.
------------------------------/etc/mail.blaccklists------------------------------ # # IMPORTANT UPDATE: As of 31 July 2001, Mail-Abuse.Org list are available by # paid subscription only. You must subscribe before using their system # ## /etc/mail.blacklists - contains blacklist domain names to access through DNS # ## MAPS Realtime Blackhole List (RBL) # # rbl.maps.vix.com # ## MAPS Dial-Up List (DUL) # # dul.maps.vix.com # ## MAPS Relay Spam Stopper (RSS) # # relays.mail-abuse.org # ---------------------------end of /etc/mail..blacklists--------------------------
During the UBE filter phase (Post SMTP) the message is actually contained in three files: the "control", "header", and "body" files. The envelope headers are contained in the control file of each message. Each file is named based on the unique message identifier which NMS assigns to it. Each type of file is denoted by a suffix to the message identifier: "-C" for control, "-H" for header, and "-B" for body. (Under NT the suffixes are "-Control", "-Header", and "-Body"). The message identifier is not the same as the RFC 822 Message-ID: header unless the message, as received, is missing the Message-ID: header, in which case, NMS will add a Message-ID: header with this value. The control files are contained in the 'control' subdirectory of the postoffice directory, and the header and body files are contained in the 'messages' subdirectory.
The envelope headers in the message "control" file correspond to the RFC 821 SMTP headers in the following manner: the User-From header maps to the "MAIL FROM:" SMTP command, and the Channel-To header maps to the "RCPT TO:" SMTP command.
The following sample control file, named "771870A910CD.AAA6DD9-C", in the '/var/spool/postoffice/control' directory, contains these lines:
Function: SMTP-Router Control-Type: Mail Priority: 2 Submitted-Date: Fri, 4 Sep 1998 04:43:02 -0500 MIME-Encoding: 7BIT Host-From: bulk-07-imc.newswire.microsoft.com [131.107.3.118] User-From: SMTP<MicrosoftHomeAdvisor_001898@news.newswire.microsoft.com> MAIL-Exts: Channel-To: SMTP <user1@mydomain.com> Channel-To: SMTP <user2@otherdomain.com> RCPT-Exts: Message-Size: 4822 MTA-Hops: 1 Trace: SMTP-Accept
Please note that there is a separate Channel-To: line for each recipient. One of the properties of the UBE filters that is not explained in the Netscape documentation is that a filter succeeds if ANY lines match the filter. So if you have a line in your filter.cfg like the following:
Channel-To: "mydomain.com" EXITthen if any recipients match "mydomain.com", the pattern match succeeds and the message is sent to ALL recipients, including in this case, <user2@otherdomain.com>. You may also notice that I have included the strings "[.@]" at the beginning and ">" at the end of the Channel-To pattern in the above filter.cfg. The reason being is that the pattern "mydomain.com" also matches the addresses <mydomain.com@somewhere.com> and <user@mydomain.com.anywhere.com>. The addition of these strings to the pattern guarantees that only the whole domain name matches at the end of the address.
There are also several other headers of interest in the control. MTA-Hops is basically a count of the Received: headers and Message-Size is the length in bytes of the message body. It is possible (or so I am told, although I haven't tested it myself) to use a pattern that checks the message size against a limit, e.g.
Message-Size:envonly "[1-9]([0-9]){6,}" REJECT "Message too large"
rejects messages if they are larger than 999,999 bytes.
Coalition Against Unsolicited Commercial Email
Forum for Responsible and Ethical Email
Consumer.Net's Network Tools
Fight Spam on the Internet!
Anti-Spam Directory of Information and Resources
UXN Spam Combat
SpamHaus - Who's behind your spam?
Sam Spade
Spam Cop
The Spam Bouncer
Geek Tools
net.demon online spam-fighting tools
Other links:
Spam filters
Secure email
Email services
Free email
Internet Yellow Pages
Games
Shareware
AI net