Programmers aim to hit spammers where it hurts

Around 500 programmers, researchers, hackers and IT administrators who gathered for the Spam Conference at the Massachusetts...

Around 500 programmers, researchers, hackers and IT administrators who gathered for the Spam Conference at the Massachusetts Institute of Technology (MIT) on Friday expressed their desire to find a spam filter so effective that spammers would receive few, if any, responses, making sending unsolicited bulk e-mail a financially prohibitive task.

"Spamming is a business, and the theft efficiency ratio is the same as stealing hubcaps," said programmer William Yerazunis.

But the high payoff for sending spam could change if an e-mail filter like the one Yerazunis pioneered became widely adopted by large Internet service providers (ISPs).

Yerazunis wrote a language for writing filters based on the Bayesian system which assigns statistical probabilities to whether or not an e-mail is spam. The language is called CRM114, and he wrote a filter program in CRM114 called MailFilter.

At the conference at least, MailFilter, which is still in alpha testing, was being seen as the great white hope for battling the escalating spam problem.

Yerazunis claimed MailFilter was 99.915% accurate in identifying spam. "I'm only 99.84 percent accurate at identifying spam, so this is much more accurate than I am."

Spam Conference organizer Paul Graham said he was extremely excited about Yerazunis' solution, saying that it looked "the most promising"..

Graham himself is a big proponent of filters based on the Bayesian system and he has written his own research report on the subject

His paper, "A Plan for Spam", released last August and posted online at, has generated a lot of discussion within the spam fighting community.

Graham has also written his own filter based on the Bayes system "I believe in filters because I personally do not have a spam problem."

Graham admitted that the idea that filters alone could thwart spam did not get serious discussion until about a year ago. However, both Graham and Yerazunis believe that if there is widespread adoption of filters that are accurate enough to make spamming economically prohibitive, the problem will cease without the need for legislation or other measures.

Yerazunis calculated that spam filters needed to be at least 99.5% accurate to push the cost of sending bulk unsolicited e-mail to about the same as it is to send direct snail mail, making it a far less attractive method for sending solicitations.

The problem is getting large ISPs to adopt the filters. As it stands now, each ISP is taking its own approach.

Still, representatives from Yahoo!, AOL and Microsoft registered for the conference and showed interest in hearing new ideas.

One of the perennial problems when employing any anti-spam system is deciding what is and what isn't spam. Whether something should be considered spam is often up to the user, and this makes building and employing filters especially tricky.

Spam-fighters are hoping to collect as much spam as possible so they can perform analysis and research on the features that make up spam.

Paul Judge, a representative for e-mail security firm CipherTrust, said his company was collecting a spam archive for this purpose. Over the past two months the company has collected 250,000 pieces alone, and is on track to have 1.5 million pieces within the first year.

"Spam messages are starting to look more and more like non-spam messages," Judge said, adding that analysis is becoming even more important.

While CipherTrust is building its spam archive, Chicago-based programmer Philip Tom was at the conference, handing out we he called "a day of spam" - a disk containing 250,000 spam e-mails.

Tom said that he has an archive of more than 50 million spam messages, and receives 250,000 a day from an undisclosed source.

While Tom said he might sell the archive for research purposes, he also thinks he might just hand it over "for the greater good" of eliminating spam.

But when Graham was asked whether he was planning another Spam Conference, given the success of this one, the answer was no. "Hopefully we will solve this problem and we won't need another conference," he said.

"I don't want to be working on the spam problem 10 years from now!"

Read more on IT risk management