#
#	$SRC_Id: README.tmpl,v 1.27 2014/12/19 15:42:57 craski-shell_86 Exp $
#


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Function
# ========
#

OpenBSD spamd blacklists via multi-protocol, retries + random delays


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Description
# ===========
#

Compliments spamd-setup by downloading resources, with up to 12 attempts,
each after a random delay of up to 5 minutes.

Protocols supported: http, ftp, scp, sftp, rsync.

Downloading tool settable; ftp, lynx, curl, rsync, [other]...

Proxies usable, caching reduces everyone's bandwidth/time/expenses.

Many hosts polling the remote servers at the same time (root's default
cron job) can result in the "zero minute rush" issue, often resulting in
"Illegal seek" or "Broken pipe" errors.

See:
http://thread.gmane.org/gmane.os.openbsd.misc/196071
http://openbsd.7691.n7.nabble.com/spamd-setup-in-crontab-td80311.html

This tool sorts out those niggles.


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Name
# ====
#

Abersnuik

The Scots word portion 'Aber' often means a town at the merging/mouth of a river.

Examples include: Aberdeen, Abernethy, Aberdour, Abertay, Abercrombie, etc.
http://en.wikipedia.org/wiki/Aber_and_Inver_%28placename_elements%29#In_Scotland

The imaginary river 'Snuik' could be one full of spam. This software establishes
a Keltic warrior settlement with watermill at Abersnuik, to grind spam to pulp.
Something like the spectacular New Lanark World Heritage textile milling village.
http://www.UndiscoveredScotland.co.uk/lanark/newlanark/
http://www.NewLanark.org/
http://en.wikipedia.org/wiki/New_Lanark
The software author recommends a stay in the hostel, with many fine walks.


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Usage
# =====
#

$OpenBSD$

+-------------------------------------------------------------------------------
| Running abersnuik-1.3.25 on OpenBSD
+-------------------------------------------------------------------------------

#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Set up a mail alias for '_abersnuik' to 'postmaster' in your MTA
# ================================================================
#

Postfix example:

$ postconf -xh alias_database
btree:/etc/postfix/alias_database.map
$ vi /etc/postfix/alias_database.map
$ sudo postalias $(postconf -xh alias_database)
$ postalias -q _abersnuik $(postconf -xh alias_database)
postmaster
$ sudo /etc/rc.d/postfix -d reload

# Test the mail routing:
$ tail -f /var/log/maillog | fgrep _abersnuik &
$ print ignore | mail -s test _abersnuik


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Set up log file rotation
# ========================
#

example 1:
$ fgrep abersnuik /etc/newsyslog.conf
/var/log/abersnuik      _abersnuik:wheel        640  7     250  *     Z

example 2 (if you have a group 'postmasters' for mail admin duties):
/var/log/abersnuik      _abersnuik:postmasters  640  7     250  *     Z


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Edit /etc/mail/spamd.conf to suit your needs. Refer to spamd.conf(5)
# ====================================================================
#

The 'method' needs to be 'exec'
The 'file' needs to be:
	'/usr/local/libexec/abersnuik (http|ftp|scp|sftp|rsync) URL'

The default downloading tool can be (optionally) set, such as:
	[/usr/bin/]ftp
	[/usr/bin/]lynx
	[/usr/bin/]scp
	[/usr/bin/]sftp
	[/usr/local/bin/]curl
	[/usr/local/bin/]rsync

For the protocols of http and ftp, unless otherwise stated, curl (if installed)
will be used as it only requests a file that has been modified later than any
cached file (even without proxy servers), reducing transfers.

Proxies can be used, also see below.

The cache directory can be set, possibly for sharing to other mail servers via
(s)ftp, scp, NFS, rdist, cron,..... [your wonderful imagination.....]
This means one host can poll the Internet and cache on behalf of other hosts.


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Example /etc/mail/spamd.conf
# ============================
#

all:\
	:uatraps:nixspam:bsdly:psbl:uceprtct:

# >[http]<
# University of Alberta greytrap hits.
# Vanilla http replacement example:
uatraps:\
	:black:\
	:msg="Your address %A has sent mail to a ualberta.ca spamtrap\n\
	within the last 24 hours":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik http www.openbsd.org/spamd/traplist.gz

# >[ftp]<
# University of Alberta greytrap hits.
# In this example, our primary mail server has obtained the file from the Internet
# and extracted it in a configured cache (/var/spool/ftp/pub/OpenBSD/spamd-cache)
# and this host duplicates it over via ftp. (See below for cache dir setting.)
# $ grep ^ftp /etc/passwd
# ftp:*:212121:212121:Anonymous FTP:/var/spool/ftp:/sbin/nologin
# Refer to: http://openbsd.7691.n7.nabble.com/FAQ-10-AnonFTP-tp241057p241851.html
uatraps:\
	:black:\
	:msg="Your address %A has sent mail to a ualberta.ca spamtrap\n\
	within the last 24 hours":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik ftp \
		mx0.example.com/pub/OpenBSD/spamd-cache/www.openbsd.org_spamd_traplist

# >[scp]<
# Nixspam recent sources list.
# In this example, our primary mail server has obtained the file from the
# Internet and extracted it in the default cache dir, so we can scp it over.
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
nixspam:\
	:black:\
	:msg="Your address %A is in the nixspam list\n\
	See http://www.heise.de/ix/nixspam/dnsbl_en/ for details":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik scp \
		mx0.example.com;/var/cache/abersnuik/www.openbsd.org_spamd_nixspam

# >[http]<
# bsdly trap list.
# http://bsdly.blogspot.co.uk/2012/05/in-name-of-sane-email-setting-up-spamd.html
# http://bsdly.net/~peter/spamd.conf
bsdly:\
	:black:\
	:msg="SPAM. Your address %A has sent spam within the last 24 hours\n\
	See http://www.bsdly.net/~peter/traplist.shtml for details":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik http www.bsdly.net/~peter/bsdly.net.traplist

# >[sftp]<
# bsdly trap list.
# In this example, our primary mail server has obtained the file from the
# Internet and saved it in the default cache, and this host gets it via sftp.
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
bsdly:\
	:black:\
	:msg="SPAM. Your address %A has sent spam within the last 24 hours\n\
	See http://www.bsdly.net/~peter/traplist.shtml for details":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik sftp \
		mx0.example.com;/var/cache/abersnuik/www.bsdly.net__peter_bsdly.net.traplist

# >[rsync]<
# Passive Spam Block List
# An rsync example.
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
psbl:\
	:black:\
	:msg="Your address %A has sent mail to a spamtrap. See http://psbl.org/faq/":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik rsync psbl-mirror.surriel.com;;psbl/psbl.txt

# >[rsync]<
# UCEPROTECT level 1 blacklist
# Another rsync example.
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
uceprtct:\
	:black:\
	:msg="Your address %A is in the UCE Protect level 1 blacklist.\n\
	See http://www.uceprotect.net/rblcheck.php?ipr=%A for details":\
	:method=exec:\
	:file=/usr/local/libexec/abersnuik rsync \
			rsync-mirrors.uceprotect.net;;RBLDNSD-ALL/dnsbl-1.uceprotect.net


# abersnuik configuration:
#
# An example using /usr/bin/ftp and a caching squid proxy on localhost
# NOTE: the shell environment's proxy takes precedence.
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
abersnuik:\
	:http_proxy=http;//localhost;3128:\
	:ftp_proxy=http;//localhost;3128:\
	:tool=ftp

#
# Another example, using /usr/bin/lynx with proxies set in /etc/lynx.cfg
# Here the cache directory is also set, and our other servers ftp from it:
abersnuik:\
	:tool=lynx:\
	:cache=/var/spool/ftp/pub/abersnuik

#
# An example, using /usr/local/bin/curl (if installed, otherwise /usr/bin/ftp)
# [:BLANK:]

#
# An example, using /usr/local/bin/curl (if installed, otherwise /usr/bin/ftp)
# and a caching apache proxy on another host
# NOTE: due to a documented bug in getcap(3), colons are replaced with semi-colons:
abersnuik:\
	:http_proxy=http;//web-proxy.example.com;8080


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Testing/debugging/development
# =============================
#

$ sudo /usr/libexec/spamd-setup -d
blacklist uatraps 107052 entries
blacklist nixspam 40000 entries
blacklist bsdly 1216 entries
blacklist psbl 381761 entries
blacklist uceprtct 512447 entries

$ tail -f /var/log/abersnuik &
$ traps=$(mktemp)
$ sudo /usr/local/libexec/abersnuik \
	http www.openbsd.org/spamd/traplist.gz > ${traps} 2>&1
$ less ${traps}

$ sudo su -l -s /bin/ksh _abersnuik /usr/local/libexec/abersnuik \
	http www.openbsd.org/spamd/traplist.gz > ${traps} 2>&1
$ less ${traps}


$ ls -loa /var/cache/abersnuik
total 11916
drwxrwxr-x  2 _abersnuik  postmasters  -         1024 Dec 19 15:34 ./
drwxr-xr-x  6 root        wheel        -          512 Dec 18 17:40 ../
-rw-r--r--  1 _abersnuik  postmasters  nodump  620343 Dec 19 15:30 psbl_mirror.surriel.com__psbl_psbl.txt
-rw-r--r--  1 _abersnuik  postmasters  nodump      18 Dec 19 15:30 psbl_mirror.surriel.com__psbl_psbl.txt.sum
-rw-r--r--  1 _abersnuik  postmasters  nodump 3409588 Dec 19 15:34 rsync_mirrors.uceprotect.net__RBLDNSD_ALL_dnsbl_1.uceprotect.net
-rw-r--r--  1 _abersnuik  postmasters  nodump      19 Dec 19 15:34 rsync_mirrors.uceprotect.net__RBLDNSD_ALL_dnsbl_1.uceprotect.net.sum
-rw-r--r--  1 _abersnuik  postmasters  nodump   19121 Dec 19 15:28 www.bsdly.net__peter_bsdly.net.traplist
-rw-r--r--  1 _abersnuik  postmasters  nodump      17 Dec 19 15:28 www.bsdly.net__peter_bsdly.net.traplist.sum
-rw-r--r--  1 _abersnuik  postmasters  nodump   19301 Dec 19 14:28 www.bsdly.net__peter_bsdly.net.traplist~
-rw-r--r--  1 _abersnuik  postmasters  nodump  881737 Dec 19 15:26 www.openbsd.org_spamd_nixspam
-rw-r--r--  1 _abersnuik  postmasters  nodump  162875 Dec 19 15:26 www.openbsd.org_spamd_nixspam.gz
-rw-r--r--  1 _abersnuik  postmasters  nodump      18 Dec 19 15:26 www.openbsd.org_spamd_nixspam.gz.sum
-rw-r--r--  1 _abersnuik  postmasters  nodump  157386 Dec 19 14:27 www.openbsd.org_spamd_nixspam.gz~
-rw-r--r--  1 _abersnuik  postmasters  nodump  457754 Dec 19 15:24 www.openbsd.org_spamd_traplist
-rw-r--r--  1 _abersnuik  postmasters  nodump  133082 Dec 19 15:24 www.openbsd.org_spamd_traplist.gz
-rw-r--r--  1 _abersnuik  postmasters  nodump      18 Dec 19 15:24 www.openbsd.org_spamd_traplist.gz.sum
-rw-r--r--  1 _abersnuik  postmasters  nodump  134207 Dec 19 14:42 www.openbsd.org_spamd_traplist.gz~


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Source code access
# ==================
#

#
# HTTP download:
#
http://web.britvault.co.uk/products/abersnuik/abersnuik-1.3.25.tar.gz

#
# Anonymous CVS checkout:
#
# *** NOTE: Neither password nor network encryption is needed.
#
you$ cd $(mktemp -d)
you$ export CVSROOT=':pserver:anoncvs@src.britvault.co.uk:/var/spool/cvs'
you$ cvs -q checkout -P abersnuik

#
# Edit files and test as per the above section.
# When done, generate a diff of your work, then
# send it with your comments to the mailing list below:
#

you$ cvs -q diff -uNp abersnuik/ | tee abersnuik.$$.diff

#
# Mailing list for discussion & diffs/patches:
#

post:	abersnuik@britvault.co.uk
join:	abersnuik+subscribe@britvault.co.uk
leave:	abersnuik+unsubscribe@britvault.co.uk


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Building
# ========
#

The script 'abersnuik' is included in the tarball/CVS repository,
which has been built by running the included './release.ksh build'


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
# Installing
# ==========
#

you$ sudo install -b -p -S -o root -g bin -m 555 abersnuik /usr/local/libexec/abersnuik
you$ sudo install -o _abersnuik -m 640 /dev/null /var/log/abersnuik
you$ sudo install -d -o _abersnuik -m 775 /var/cache/abersnuik


#
# =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
#