I seem to have blogged a lot of stuff about mail lately… Now that I’ve found it again, I wanted to link to Justin Hawkins’ useful Mail Filtering stuff, which uses procmail and a couple of other scripts to automagically archive inbound email. Very cool.
Having worked for ISP’s for much of my working life, I know about mail. That is, about receiving it. Lots of it. It never stops. Human beings send me a fair bit, machines send me a lot more.
Over the years I’ve experimented a lot with filtering mail. I consider my current setup quite usable, and I’ve documented most of what I’ve done here, in the hope that someone else might find it useful as well.
This paper outlines a new method for spam control which the authors have called Greylisting. I haven’t read the whole thing yet, but it looks interesting.
Greylisting has been designed from the start to satisfy certain criteria:
1. Have minimal impact on users
2. Limit spammers ability to circumvent the blocking
3. Require minimal maintenance at both the user and administrator level
Greylisting could be a good way of bridging the gap until SPF can be widely implemented. I still beleive SPF is a good long term result because it makes the forgery of From addresses extremely difficult, hence forcing spammers to use fixed IPs and domains which can be easily filtered.
After reading the analysis of Sobig you start to get the impression that some viruses these days are professionally written. SoBig almost seems to look like a traditional iterative development process where you release an initial version, improve it, release the next version and repeat until perfection is acheived.
Sobig started out with a fixed From: address, Sobig.b changed the From address to something we couldn’t block quite so readily. Later versions used faked from addresses to make blocking even harder. Over time SoBig has become more sophisticated at mass mailing itself, and better at hiding its stage 2 payload. SoBig even seems to have a timer which kills off the old version, presumably to stop it from competing with the next one. And most importantly, SoBig has spread faster and further each time.
We haven’t had a new SoBig for a while though, but we did get a “MyDoom”. MyDoom seems to have been a lot more sucessful than SoBig ever was. Instead of receiving 1 or 2 copies through our mail server before out anti-virus program would detect it, we got hundreds. At its peak, MyDoom was well over half of our email traffic.
Now Information Week is reporting that the Slashdot
After the Spam FixIt at LCA the other week I had this neat idea (which I didn’t get around to writing about) about using something like a reverse MX record to identifiy authorised mail servers for a given domain. As it turns out, theres already a proposal documented for doing exactly that, called SMTP Sender Permitted From (SPF).
What’s the idea here?
Domains already publish MX records to tell the world what machines receive mail for the domain.
Domains can also publish “reverse MX” records to tell the world what machines send mail from the domain.
The world can check those records to make sure mail isn’t spoofed.
With SPF, those “reverse MX” records are easy to publish: one line in DNS is all it takes.
I’ve now seen two of these messages in the last couple of hours. In one case I can say that the from address is definately forged, in the other I’m not sure. In both cases the subject line and message content were virtually identical (except for the string of random characters in the message body. MIMEDefang stripped .exe file attachments off of both messages.
-----Original Message-----
From: fraser@trilobytes.com.au
Sent: Monday, 19 January 2004 13:39
To: linuxsa@linuxsa.org.au
Subject: Hi
Test =)
vrrtlwkh
--
Test, yep.
===========================================================================
WARNING: This e-mail has been altered by MIMEDefang. Following this
paragraph are indications of the actual changes made. For more
information about your site's MIMEDefang policy, contact
Jim O'Halloran
. For more information about MIMEDefang, see:
http://www.roaringpenguin.com/mimedefang/enduser.php3
An attachment named vlpfmr.exe was removed from this document as it
constituted a security hazard. If you require this document, please contact
the sender and arrange an alternate means of receiving it.
In the case of the message above the forgery was good enough to get past the list server on our local Linux group’s mailing list.
I’m guessing I’m seeing the next Windows virus outbreak in its infancy. Has anyone else seen messages like this lately?
UPDATE: Acording to Symantec this is the W32.Beagle.A@mm worm.
W32.Beagle.A@mm is a mass-mailing worm that will only work until 28th of January. This worm will insert several files and registry keys on the system. It will also access remote websites, and email all contacts it can find.
The second Linux.Conf.Au fixit that I want to was the Spam fixit. This was hosted by Tridge and Rusty, although somehow Rusty managed not to be there. There was a lot of lively discussion, as everyone has an opinion on Spam these days.
Initially DCC and Razor type systems were showing a good amount of promise, but these use strong checksums, which are easily defeated by inserting random words into the message. Loose checksums are less easily defeated in this way, but instead can produce higher false positive rates.
Penny black techniques were discussed, which are essentially a requirement that the sender perform a computationally intensive operation before sending the message. The sender can then quickly verify the result, and accept the message. The problem with this technique is that it disadvantages normal mail users more than it does spammers. It’ll slow down the rate of spam, but spammers can simple add more trojaned Windows boxes to their network and we’re back to the same old problem.
Someone mentioned the sendmail tool “milter sender” which looks very interesting. Milter sender verifies the from address of a message on receipt, and can cause the message to be rejected if the from address is invalid.
One attendee also mentioned that he’d tweaked Spamassasin to assign a slight negative score to messages originating in Australia and a slight positive score to messages originating in “spammy” countries like Korea, Brazil and Nigeria.
Today was a more hands on introduction to IPv6. John Barlow the Advanced Communications Services Coordinator for AARnet gave all of today’s presentations and started out the day with “IPv6 101 Hands On”. This in some respects was a rehash of some things covered yesterday, but it did cover some more technical type issues.
One interesting comment made was that NAT has a happy side effect of enhancing security (which I know and exploit regularly). However, John tells us that NAT can be hacked to get at machines behind the NAT gateway, which is something I really need to investigate further.
The basic structure of the IPv6 address space is as follows… The IPv6 equivalent of 127.0.0.1 is ::1. Most globally routable IPv6 IP’s start with 2001: and current best practice dictates that these are the only IPs to be routed across the backbone. IPv6 IP’s starting with fe80: indicate a “link local” address. Link local addresses are used for bootstrapping, autoconfiguration, etc and should not be propagated beyond a router.
Fec0::/10 are site local addresses (similar to the private IP ranges in IPv4) ut the use of these has been or will be deprecated shortly. :FFFF:a.b.c.d are mapped IPv4 addresses, and don’t actually exist, but might show up in logs. ::a.b.c.d addresses are used to arrange tunnels. There are more prefixes allocated, but many are not currently used, so the above are the commonly seen IPv6 addresses.
Unlike DHCP, IPv6 autoconfiguration does not assign name servers, time servers, etc. Other solutions such as anycast addressing, etc can be used to work around this, but no mechanism has been standardized at this stage.
IPv6 uses a simpler header than IPv4, many un used, or less commonly used fields are dropped or moved into extension headers. An IPv4 header is a fixed 40 bytes, with important fields aligned on 64 bit boundaries for faster processing on 64 bit architectures. Extension headers are a fixed 64 bits each in length and follow the main header. One of the design goals of IPv6 was to ensure that routers do minimal processing of packets. IPv6 packets can’t be fragmented in transit, which makes ICMP MTU discovery mandatory.
A workshop session followed, during which we were walked through IPv6 setup on our own machines. The demo was done on Linux. However the WiFi coverage where I was sitting in the Lecture Theatre was pretty spotty (more off than on), so I wasn’t able to get much going, gave up and went to lunch.
After lunch John finished off the workshop session by showing an IPv6 router setup on Linux. There is a daemon called radvd (Router Advertisement Daemon) which handles the router multicast packets required for autoconfiguration. Quogga (used to be Zebra) handles the BGB, etc required for serious internet routing.
Next up was a session on Provider Independent Addressing (PIA). From the sounds of things this is now pretty much dead in the water, and in fact current best practices will prevent PIA packets from being routed.
Essentially PIA allocates a /48 network to every 10 x 10 metre square of the earths surface (including oceans). The network prefix is mathematically derived from the Latitude and Longitude coordinates of your current locations. However there are no algorithms for resolving conflicts (e.g. multiple floors of a building, or an aircraft flying overhead), and in fact the suggested remedy was to “pick some random suare of ocean and use that” in the event of conflicts.
We then covered IPv6 Global Routing. Essentially obtaining an IPv6 address block is simple. You just go to APNIC and commit to handing out 200 /48’s to customers in the next two years. However, most of us will simply obtain our allocations from our ISP, but this means that you can’t take your allocation with you when you change ISP’s. However the old days of getting your own class C and having it routed via any ISP are over with IPv6. To control the growth of the routing tables IPv6 address space is being allocated only to ISPs, hence the requirement to allocate 200 networks above.
Finally we had an “Issues with IPv6” presentation which covered some of the issues in an IPv6 migration.
Autoconfiguration will mean that Dynamic DNS will become almost a necessity for mot organizations. However, most common DNS servers (e.g. BIND) will handle IPv6 addresses.
Application support is pretty good, but traditional network file sharing protocols such as SMB and NFS are yet to be upgraded to IPv6.
The Linux IPv6 HOWTO is updated regularly and is an excellent source of information related to IPv6 and migration issues.
IPv6 should run over pretty much any Layer 2 hardware (e.g. hubs, network cards, wireless gear) without problems. Most Layer 3 devices (such as routers) should only need a software upgrade in order to deal with IPv6, but some switches and routers which use hardware acceleration to process packets and therefore may need to be replaced before they achieve full throughput/functionality.
Because all IPv6 addresses are provider assigned multihoming for reliability becomes awkward and usually becomes something that needs to be negotiated with the ISP’s concerned. In theory it should work, but in practice its difficult.
There is also a thing called NAT-PT which is an IPv6 NAT into IPv4 and back again. As per traditional NAT each protocol needs to be handled individually. NAT-PT is achieved via a DNS Server hack and some additional bindings on the IPv6 interface. Its pretty nasty, but is a reasonable migration tool.
All slides from today’s (and some of yesterdays) presentations can be found on AARNet’s site. Overall I’ve learnt a lot from the IPv6 MiniConf, and its been a pretty worthwhile experience. The main conference starts tomorrow, can’t wait.
Just finished up at the first day of Linux.Conf.Au 2004… Well, the first day of the mini-conferences anyway. The real conference gets under way on Wednesday morning. I spent today in the IPv6 mini-conference. I didn’t know much about IPv6 before, and a reasonable amount of what I heard today went over my head too. However, I do know a good bit more about IPv6 than I did this morning, so that’s a good start…
Leading off the MiniConf was a presentation from Trent Lloyd, who gave a general introduction and overview of IPv6. Essentially IPv6 is the next version of TCP/IP, designed to resolve some of the problems caused by a shortage of IPv4 address space. IPv6 features a 128bit IP Address giving a greatly expanded address space compared to IPv4 (32 bits), An IPv6 address is written as 8 16 bit hexadecimal sections (e.g. FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF:FFFF), and there are a few conventions for shortening down IP addresses when written (e.g. :0000: can be written as simply ::, and :0035: can be written as :35:). Another feature of IPv6 is that it can be encrypted (via IPSec) on the wire, leaving application level protocols less vulnerable to sniffing.
IPv6 is designed to be self configuring, with no DHCP required. To achieve self configuration, a router advertises a prefix to the network, and the hosts on the network combine the prefix with their MAC address to create a unique IP. However, using the MAC address poses some privacy issues because it allows a machine to b tracked across different networks. For this reason some IPv6 stacks can also create random or “disposable” IP addresses instead.
Tunnels allow IPv6 packets to be transferred across IPv4 networks with no provider support for IPv6 required. However this leads to inefficient routing and high latency. 6to4 gateways also allow IPv6 packets to be transported across IPv4 networks, but instead works as kind of a NAT gateway between the two IPv6 address spaces. 6to4 works in part by encoding an IPv4 address in full into the IPv6 address of a host.
Application support for IPv6 is fairly wide spread, with most versions of Windows having IPv6 support in some form, as well as Linux and the *BSD’s. IPv6 is also supported by most applications such as IE, Mozilla, Apache and some IRC and FTP servers.
IPv6 addresses are stored in the DNS as an “AAAA” record instead of the more usual “A” record. Reverse lookups on IPv6 addresses are achieved via .in-addr.ipv6 zones rather than the usual .in-addr.arpa zones. Because of the length of IPv6 addresses, DNS becomes pretty much mandatory as the new addresses will be too long to readily memorise.
One oft stated advantage of IPv6 which was mentioned several times during the day is that having such a huge address space means an end to NAT, but this carries the penalty of requiring greater host security (no more hiding insecure Windows boxes behind a NAT gateway).
The slides from this talk can be found at http://www.sixlabs.org/talks/
Hesham Soliman was the next up and covered Mobile IPv6 in a Wireless internet. The main driving thrust behind IPv6 adoption will come from the desire to network all of our mobile devices, and mobile data services into the future (i.e. mobile wireless broadband services). Carriers are moving towards IP based cellular networks (e.g. CDMA 2000 in the states) for mobile data services, and there are far more mobile phones in the world than there there will ever be IPv4 addresses for. This pressure will probably lead to carriers implementing IPv6.
Mobile IPv6 allows a machine to move from one network to another without breaking any socket connections which are active at the time (provided of course the physical connection is never broken), e.g. in a hand off from one cell to the next in a mobile network. Essentially this works by having a “home address” which any machine you connect to will see. Any packets to/from that home address are tunneled to a “care of” address which is your physical IP address at the time. Thus when a machine moves from one IP to the next, the car of address changes, but the home address remains constant, thus connections are maintained. There is security covering the registration f the tunnels from home address to care of address, to ensure that Mobile IP can’t be manipulated for Denial of Service purposes.
The last presentation before lunch was given by Ahmet Sekerocoly (probably mispelt your last name, sorry I couldn’t read it on the slides) and covered the simulation of IPv6 networks using a package called OMNet++. OMNet is an open source package (GPL like license) which allows simulation of large scale networks. This is probably of great interest to academics and large phone network operators, but was a little esoteric for most of us, so I didn’t take many notes.
After lunch John Barlow covered the AARnet tunnel broker service. Tunneling requires a tunnel server and some special software on your own machine. Once set up the software will establish an IPv6 in IPv4 tunnel between you and the tunnel broker, which will in turn route or tunnel the traffic on its way to its final destination.
Firewalls commonly block tunneled packets, IPv6 firewalls are fairly new. Some features in which might be logical in an IPv6 firewall probably don’t exist yet (i.e. blocking all packets which aren’t encrypted and authenticated).
In tunneling an IPv6 packet is wrapped in an IPv4 packet, shoved into the tunnel and routed across the IPv4 network, where it appears out the other end of the tunnel. Because it is routed from one end of the tunnel to the other as an IPv4 packet, the IPv6 packet we’re tunneling may not take the most efficient route across the internet. Inevitably this leads to higher latency and delays.
A 6to4 gateway creates an automagic connection between IPv6 hosts with an intermediate IPv4 network. It works in part by encoding an IPv4 address into the IPv6 destination address. The packets then go to a 6to4 relay router which sends them across the IPv4 where the packet hits another 6to4 relay and pops out the other end as IPv6 to be routed on to its final destination. There is a list of public 6to4 relay routers Nick Sayer’s site , but be warned the site doesn’t like IE.
Windows XP has IPv6 support out of the box, which is activated by simply typing “ipv6 install” from a command prompt. Other versions of Windows also support IPv6, but patches are available from Microsoft or third parties.
The easiest way to test IPv6 connectivity is to go to www.karme.net … If you’ve connected via IPv6 the turtle logo at the top of the page will dance. There is also a ping6 utility on most systems which will ping an IPv6 host. Ping is overloaded on some systems to ping both IPv4 and IPv6 hosts.
Finally, Michael Biber wrapped up the day with “Implications of IPv6 in Oz”. Effectively the presentation had very little to with IPv6 in Australia, other than the message that “IPv6 is big in Asia so Australia needs to jump on board early if we don’t want to be left behind”. Some interesting tidbits came out of the talk though, some of which are summarised below… Japan, Korea a China are all driving IPv6 adoption, and IPv6 support is essentially required in order to sell product (routers, etc) into those countries.
China especially has a reasonably large and growing internet population but a total allocation of about 9 million IP addresses. In contrast some early adopters of the internet have IP Address allocations totally out of proportion with their user base (e.g. MIT has 17 million IP’s, and IBM has 33 million, while the US Government has 168 million IP’s available). The US typically being the “home” of th internet and the first of the early adopters typically has a fairly large allocation of IP’s and is in no hurry to change to IPv6. Australia, also being a fairly early adopter did well with its initial IPv4 allocation, wich may explain a very limited deployment of IPv6 in this country.
Countries such as India which in some cases is buried in 5 levels of NAT to fit its user population into its IPv4 allocation are very keen to see IPv6 implemented.
CAIDA (can’t remember what that stands for) tracks versios statistics on IPv6 adoption and routing table growth as well as providing a more general “Internet Weather Report”.
Michael also took the opportunity to “soft launch” the IPv6 Forum in Australia, which will seek to promote IPv6 and related issues in Australia. The IPv6 Forum and could be very useful in raising awareness of IPv6 in this country.
Interestingly enough, 3 out of the 5 presenters today were running some form of Windows on their laptops. It could be because the IPv6 support in XP is so good, or it might reflect the fact that IPv6 is a global internet protocol, noi a Linux centric thing (or possibly both), but it was a little funny to see at a Linux conference.
All in all I learnt a fair amount, which made the day worthwhile. IPv6 isn’t on my immediate horizon, but it’s a topic I feel that’s worth keeping in touch with. Over lunch, I also had an interesting conversation with the Network Manager from QUT swapping IT war stories. All up, it was a good day. Tomorrow is the IPv6 workshop day, which should be more hands on than today, then its into the main conference on Wednesday.
I’f anyone else is blogging from Linux.Conf.Au, please leave a comment or a trackback on this post. I’ve checked feedster, and there doesn’t seem to be anything there yet (could be too early), but I’ll be interested to read other reports from the conference.
The Atlanta Journal-Constitution has an interesting profile of a small time spammer.
In the small Louisiana city of Slidell, Flo Fox feeds the hungry by day but spams by night.
The graying grandmother in a “What Would Jesus Do?” T-shirt proudly recalls stretching two turkey carcasses into enough gumbo to feed 100 of the city’s poor.
Interesting reading.
Security Focus has a detailed analysis of a web server compromised by Spammers.
One day I noticed that one of my remote servers was sending 24 hours a day a continuous 11Kbytes stream, using the 100% of the upload bandwidth (128Kbits). This specific server is running Apache and also it acts as a mail server, but, no other network application that could send during the entire day so many traffic, was installed. So, I immediately logged into my remote machine to know what was happening, thinking that my remote box was participating in any DDoS attack, but I was totally wrong. A process list (ps -ef) would open my eyes:
It seems that the machine was compromised via a poorly written piece of PHP code (GeekLog), then used to send spam. Evil, Evil stuff.
