What is Vipul's Razor?

Vipul's Razor is a distributed, collaborative, spam detection and filtering network. Through user contribution, Razor establishes a distributed and constantly updating catalogue of spam in propagation that is consulted by email clients to filter out known spam. Detection is done with statistical and randomized signatures that efficiently spot mutating spam content. User input is validated through reputation assignments based on consensus on report and revoke assertions which in turn is used for computing confidence values associated with individual signatures.

 

 

Install amavisd-new, SpamAssassin

 

apt-get install amavisd-new spamassassin clamav clamav-daemon zoo unzip bzip2 libnet-ph-perl libnet-snpp-perl libnet-telnet-perl nomarch lzop pax

 

 

설치후 의존성 때문에 AppArmor가 설치되므로 처음에 했던데로 제거를한다:

1. /etc/init.d/apparmor stop
   update-rc.d -f apparmor remove
   apt-get remove apparmor apparmor-utils

 

amavisd-new의 설정은 /etc/amavis/conf.d 에 위치

아래는 세 개의 주요 파일을 수정하지만, 해당 디렉토리의 설정 파일을 하나씩 살펴보길 바란다.

 

vi /etc/amavis/conf.d/15-content_filter_mode

@bypass_virus_checks_maps 와  @bypass_spam_checks_maps 의 커맨트를 푼다.

 

vi /etc/amavis/conf.d/20-debian_defaults

 spam-/virus-mails을 위한 spam settings and the actions 을 살펴보고 필요하면 수정해 사용하다. 아래는 기본값이다.

$QUARANTINEDIR = "$MYHOME/virusmails";

$quarantine_subdir_levels = 1; # enable quarantine dir hashing
$log_recip_templ = undef; # disable by-recipient level-0 log entries
$DO_SYSLOG = 1; # log via syslogd (preferred)
$syslog_ident = 'amavis'; # syslog ident tag, prepended to all messages
$syslog_facility = 'mail';
$syslog_priority = 'debug'; # switch to info to drop debug output, etc
$enable_db = 1; # enable use of BerkeleyDB/libdb (SNMP and nanny)
$enable_global_cache = 1; # enable use of libdb-based cache if $enable_db=1
$inet_socket_port = 10024; # default listening socket
$sa_spam_subject_tag = '***SPAM*** ';
$sa_tag_level_deflt = 2.0; # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.31; # triggers spam evasive actions
$sa_dsn_cutoff_level = 10; # spam level beyond which a DSN is not sent
[...]
$final_virus_destiny = D_DISCARD; # (data not lost, see virus quarantine)
$final_banned_destiny = D_BOUNCE; # D_REJECT when front-end MTA
$final_spam_destiny = D_BOUNCE;
$final_bad_header_destiny = D_PASS; # False-positive prone (for spam)

 

• Amavis 설정에 대해서는 Amavisd-new 로 Spamassassin 관리 를 참조.

 

 

vi /etc/amavis/conf.d/50-user

파일의 중간 쯤에 $pax='pax'; 를 추가한다.

 

vi /etc/amavis/conf.d/05-node_id

1. $myhostname = "your_domain";

 

 

clamav사용자, 그룹 추가후 시작

1. adduser clamav amavis
   /etc/init.d/amavis restart
   /etc/init.d/clamav-daemon restart
   /etc/init.d/clamav-freshclam restart

 

 

 

Postfix에 추가

 

Postfix to pipe incoming email through amavisd-new:

1. postconf -e 'content_filter = amavis:[127.0.0.1]:10024'
   postconf -e 'receive_override_options = no_address_mappings'

 

vi /etc/postfix/master.cf

아래 라인을 추가해 준다.

master.cf

amavis unix - - - - 2 smtp

-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes

127.0.0.1:10025 inet n - - - - smtpd
-o content_filter=
-o local_recipient_maps=
-o relay_recipient_maps=
-o smtpd_restriction_classes=
-o smtpd_client_restrictions=
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o mynetworks=127.0.0.0/8
-o strict_rfc821_envelopes=yes
-o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
-o smtpd_bind_address=127.0.0.1

 

 

 

Postfix 를 재시동한다.

1. /etc/init.d/postfix restart

 

 

1. netstat -tap

 

아래와 같이 Postfix (master) listening on port 25 (smtp) and 10025, and amavisd-new on port 10024:

1. tcp        0      0 localhost:10024         *:*                     LISTEN      915/amavisd (master
   tcp        0      0 localhost:10025         *:*                     LISTEN      1993/master    
   tcp        0      0 gtko-ubuntu:mysql       *:*                     LISTEN      29433/mysqld   
   tcp        0      0 *:smtp                  *:*                     LISTEN      1993/master

 

 

Testing amavisd host and port

1. $ telnet localhost 10024
   

   220 [127.0.0.1] ESMTP amavisd-new service ready
   EHLO localhost
   250-[127.0.0.1]
   250-VRFY
   250-PIPELINING
   250-SIZE
   250-ENHANCEDSTATUSCODES
   250-8BITMIME
   250-DSN
   250 XFORWARD NAME ADDR PROTO HELO
   QUIT
   221 2.0.0 [127.0.0.1] amavisd-new closing transmission channel

 

 

Testing the dedicated Postfix smtpd-daemon

When Postfix was reloaded, the new, dedicated smtpd-daemon (127.0.0.1:10025) should have been activated. A successful connection looks like this:

1. $ telnet 127.0.0.1 10025
   

   220 mail.example.com ESMTP Postfix (2.3.2)
   EHLO localhost
   250-mail.example.com
   250-PIPELINING
   250-SIZE 40960000
   250-ETRN
   250-STARTTLS
   250-AUTH PLAIN CRAM-MD5 LOGIN DIGEST-MD5
   250-AUTH=PLAIN CRAM-MD5 LOGIN DIGEST-MD5
   250-ENHANCEDSTATUSCODES
   250-8BITMIME
   250 DSN
   QUIT
   221 2.0.0 Bye

 

 

 

test-messages/sample-nonspam.txt from the amavisd test-messages to send an e-mail:

1. $ telnet localhost 10024
   

   220 [127.0.0.1] ESMTP amavisd-new service ready
   HELO localhost
   250 [127.0.0.1]
   MAIL FROM: <>
   250 2.1.0 Sender OK
   RCPT TO: <postmaster>
   250 2.1.5 Recipient postmaster OK
   DATA
   354 End data with <CR><LF>.<CR><LF>
   From: virus-tester
   To: undisclosed-recipients:;
   Subject: amavisd test - simple - no spam test pattern
   
   This is a simple test message from the amavisd-new test-messages.
   .
   250 2.6.0 Ok, id=30897-02, from MTA([127.0.0.1]:10025): 250 2.0.0 Ok: queued as 079474CE44
   QUIT
   221 2.0.0 [127.0.0.1] amavisd-new closing transmission channel

 

The maillog shows the delivery path. Here's an excerpt from a successful delivery process:

1. Nov 1 11:28:10 mail postfix/smtpd[30986]: connect from localhost[127.0.0.1]
   

   Nov 1 11:28:10 mail postfix/smtpd[30986]: 079474CE44: client=localhost[127.0.0.1]
   Nov 1 11:28:10 mail postfix/cleanup[30980]: 079474CE44: message-id=<20061101102810.079474CE44@mail.example.com>
   Nov 1 11:28:10 mail postfix/qmgr[20432]: 079474CE44: from=<>, size=822, nrcpt=1 (queue active)
   Nov 1 11:28:10 mail amavis[30897]: (30897-02) Passed BAD-HEADER, <> -> <postmaster>, quarantine: badh-le5gjszxowBk, mail_id: le5gjszxowBk, Hits: -1.76, queued_as: 079474CE44, 39505 ms
   Nov 1 11:28:10 mail postfix/smtpd[30986]: disconnect from localhost[127.0.0.1]
   Nov 1 11:28:10 mail postfix/local[30987]: 079474CE44: to=<postmaster@example.com>, relay=local, delay=0.27, delays=0.14/0.05/0/0.08, dsn=2.0.0, status=sent (delivered to mailbox: postmaster)
   Nov 1 11:28:10 mail postfix/qmgr[20432]: 079474CE44: removed

 

If the test fails, the following questions may help to debug the problem:

 

 

Install Razor, Pyzor And DCC And Configure SpamAssassin

Razor, Pyzor and DCC are spamfilters that use a collaborative filtering network.

 

1. apt-get install razor pyzor

 

DCC isn't available in the Ubuntu 8.10 repositories, so we install it as follows:

 

1. cd /tmp
   wget http://www.dcc-servers.net/dcc/source/dcc-dccproc.tar.Z
   tar xzvf dcc-dccproc.tar.Z
   cd dcc-dccproc-1.3.104
   ./configure --with-uid=amavis
   make
   make install
   chown -R amavis:amavis /var/dcc
   ln -s /var/dcc/libexec/dccifd /usr/local/bin/dccifd

    

   cd /tmp

2. wget http://launchpadlibrarian.net/11564361/dcc-server_1.3.42-5_i386.deb
   wget http://launchpadlibrarian.net/11564359/dcc-common_1.3.42-5_i386.deb
   dpkg -i dcc-common_1.3.42-5_i386.deb
   dpkg -i dcc-server_1.3.42-5_i386.deb

 

 

 

SpamAssassin이 필터 프로그램, dcc, pyzor, razor을 사용하게 설정:

 

vi /etc/spamassassin/local.cf

아래를 local.cf 에 추가해 준다.

1. #dcc
   

   use_dcc 1
   dcc_path /usr/local/bin/dccproc
   
   #pyzor
   use_pyzor 1
   pyzor_path /usr/bin/pyzor
   
   #razor
   use_razor2 1
   razor_config /etc/razor/razor-agent.conf
   
   #bayes
   use_bayes 1
   use_bayes_rules 1
   bayes_auto_learn 1

 

SpamAssassin에서 dcc 플러그인을 사용 가능으로 변경

vi /etc/spamassassin/v310.pre

loadplugin Mail::SpamAssassin::Plugin::DCC 라인의 커맨트를 푼다.

check SpamAssassin configuration:

1. spamassassin --lint

 

amavisd-new afterwards:

1. /etc/init.d/amavis restart

 

SpamAssassin rulesets 갱신:

1. sa-update --no-gpg

 

rulesets 갱신을 cron job에 추가 crontab -e

이것은 룰셋을 4.23h 마다 갱신한다.

1. sudo crontab -e

   23 4 */2 * * /usr/bin/sa-update --no-gpg &> /dev/null

crontab -l

1. m h dom mon dow command
   23 4 */2 * * /usr/bin/sa-update --no-gpg &> /dev/null

 

 

 

 

 

Quota Exceedance Notifications

If you want to get notifications about all the email accounts that are over quota, then do this:

1. cd /usr/local/sbin/
   wget http://puuhis.net/vhcs/quota.txt
   mv quota.txt quota_notify
   chmod 755 quota_notify

vi /usr/local/sbin/quota_notify

이 파일의 아래쪽 두 라인에 %를 추가해 준다.

1. [...] my $POSTFIX_CF = "/etc/postfix/main.cf"; my $MAILPROG = "/usr/sbin/sendmail -t"; my $WARNPERCENT = 80; my @POSTMASTERS = ('postmaster@yourdomain.tld'); my $CONAME = 'My Company';

   my $COADDR = 'postmaster@yourdomain.tld';
   

   my $SUADDR = 'postmaster@yourdomain.tld'; my $MAIL_REPORT = 1; my $MAIL_WARNING = 1; [...] print "Subject: WARNING: Your mailbox is $lusers{$luser}% full.n"; [...]

              print "Your mailbox: $luser is $lusers{$luser}% full.nn"; [...]
             

cron job에 쿼타 알림을 추가하려면

crontab -e

1. 0 0 * * * /usr/local/sbin/quota_notify &> /dev/null

 

참조

http://www.howtoforge.com/virtual-users-domains-postfix-courier-mysql-squirrelmail-ubuntu9.04-p3

ClamAV Upgrade and db update

 

http://www.freespamfilter.org/FC4.html