Alan Storm is a human being living in Portland, OR by way of Seattle, WA by way of Portland, OR by way of Rochester, NY. He likes making websites, and talks about that here.

He also likes to make things on the web. If you need something made on the web, drop him a line.

We're a little worried about his penchant for slipping into the third person narrative form.

Follow the Feed

Follow the Twitter

Greylisting

My web and email host has been upgrading their junk email filtering offerings and the difference is nothing short of phenomenal.

Up until now they offered a custom install of Spam Assassin 2.63 that would tag email messages with a spam score. This was useful, but ultimately a loosing proposition since smarter junk email “providers” could just test their messages against the stock Spam Assassin filters and tweak their scores lower.

So, that being the case, Spam Sieve has been handling the majority of my filtering needs. It’s a great program, but not without a few problems.

  1. Because it’s a client side program, I still need to download all the spam I receive, which was bordering on 300-400 messages a day.

  2. Due to the characteristics of my particular spam mail, Spam Sieve has the habit of occasionally marking auto-responders from commercial sites as spam. Things like Priceline and Amazon receipts. The first time I use any new web site, I need to carefully scan my spam folder for the expected auto-responder.

  3. The above is complicated by the fact that I received so much spam, parsing through 300 - 400 messages looking for legitimate mail is tedious and ultimately futile.

  4. Because Spam Sieve auto-trains, if you fail to catch a mistake it’s going to be more likely to make the same mistake in the future.

  5. I’m still one of those fools who uses POP (rather than IMAP), so if I need to remotely check my email I ssh to Pair’s servers. This means I see all the pre-filtered mail, which makes finding legit messages difficult.

Before Pair’s update, I’d been considering permanently blocking some older email addresses that I’d foolishly posted online in an unprotected format. Fortunately, the new greylisting feature means 80% of the junk mail that used to enter my inbox has just gone away.

To understand greylisting you need to know about a key feature of SMTP, the protocol used to send email messages. Like most smart internet technology, SMTP has built in procedures to handle failures. If I try to send you email and your email server is unavailable, my email server will attempt to send the message several more times before giving up.

It turns out that the majority of unwanted bulk email is sent with software that doesn’t obey the rules of SMTP. These bulk mailing programs use a methodology that’s informally referred to as “fire-and-forget”. These programs attempt to send a message once and if it fails to get through no attempt is made to resend the message.

What greylisting does is anytime the mail server sees a message from a new email server, it will automatically reject the message. Email servers (“MTAs”) that follow the rules will attempt to resend the message. Fire-and-forget servers will not. This also has the added benefit of stopping most virus email messages as well, since they’re even less likely to follow the SMTP specifications.

Now, this certainly doesn’t stop all bulk email, but if my inbox is any barometer it certainly is effective. Additionally, I’m better able to train my client-side bayesian filter (Spam Sieve) since I don’t have a spam folder that’s filled with 300 - 400 message daily. It will be interesting to see how effective this approach is long term. Smarter spammers will, as always, adapt, but the fly-by-night hacks will find it increasingly difficult to get their pills porn and poker email through. That can only be a good thing.

Links:

Greylisting White Paper

Originally published May 23rd, 2005