Jim O’Halloran’s Weblog

Published August 20th, 2005 by Jim O'Halloran

Fighting Comment Spam with Firewalls

Although my blog has been devoid of fresh content for the last few months, one thing that hasn’t changed has been comment spam. I’ve spent a considerable amount of time keeping the blog as spam free as possible, and stuck a few problems along the way.

The bigest one has been the size of the blacklist used by mt-blacklist. It seems that the version of mt-blacklist I’m using (1.6.5) has issues when the blacklist reaches a certain size. At around 4020 entries, mt-blacklist will no longer write to the blacklist, making adding new URL’s and despamming very difficult.

Another issue I found that via some simple log analysis, comment and trackback spam were a very significant portion of my website traffic (40-50%). So I decided it was time to take a more drastic approach. Here’s what I did:

Extract the IP addresses of all comment and trackback submissions from my Apache logs:

cat /var/log/httpd/jim-access_log* | grep mt-comments.cgi | grep POST | cut -f1 -d" " > comment_ips.txt
cat /var/log/httpd/jim-access_log* | grep mt-tb.cgi | grep POST | cut -f1 -d" " >> comment_ips.txt

Using a small perl script (code in the extended entry), I extracted the 20 worst offenders:

cat comment_ips.txt | ./comment_ip_sum.pl | head -n 20

Finally, I created a custom chain in iptables on my web server, and added rules to DROP all packets from the worst of the spammers. Interesting to note that each of the Top 5 IPs were posting about at least 10x the amount of spam as the remaining IPs on the list.

iptables -N web_filter
iptables -A web_filter -S  -j DROP
iptables -A web_filter -j ACCEPT
iptables -I INPUT 1 -p tcp –dport 80 -j web_filter

I could probably automate the creation and maintenance of firewall rules a little more, but that’s good enough for me.

The perl script I use and the current block list are in the extended entry to this post.

Perl script to summarise IP list.

#!/usr/bin/perl -w

$IPs = '';

while (defined($ThisLine = )) {
        chomp($ThisLine);
        $IPs{$ThisLine}++;
}

foreach $ThisOne (sort {$IPs{$b}<=>$IPs{$a} } keys %IPs) {
        printf(”$ThisOne                “. $IPs{$ThisOne}.”\n”);
};

The current block list….

134.76.10.66
148.244.150.0/24
164.100.104.146
167.206.79.225
167.206.79.226
192.55.214.159
193.136.157.2
193.205.153.238
200.106.160.70
200.135.240.27
200.196.101.98
200.213.82.131
200.30.79.126
201.147.199.205
202.175.234.163
202.28.204.123
202.28.204.143
202.64.21.172
203.144.216.211
203.166.99.0/24
203.172.200.210
205.148.35.18
207.232.201.69
207.234.146.67
207.248.240.0/24
211.75.91.2
211.75.91.3
212.0.138.91
212.175.113.52
216.136.2.253
217.67.112.98
221.186.246.66
222.183.144.199
24.97.173.130
38.118.3.16
62.145.48.18
62.193.231.242
62.245.167.6
63.230.254.28
63.230.254.29
63.97.24.6
64.50.105.227
65.165.84.11
69.50.182.154
81.240.255.226

| Post in My Blog, Blogging |