Filtering spam: Researchers propose new method to rid inboxes of unwanted email

Nov 20, 2012

Spammers have recently turned high-tech, using layers of images to fool automatic filters. Thanks to some sophisticated new cyber-sleuthing, researchers at Concordia University's Institute of Information Systems Engineering are working toward a cure.

Once upon a time, Spam came in a can and could be easily avoided. Nowadays, spam plagues email inboxes around the world, hawking miracle pills and enticing the gullible with tales of offshore bank accounts containing untold fortunes.

These once text-based email infiltrators have recently turned high-tech, using layers of images to fool automatic filters. Thanks to some sophisticated new cyber-sleuthing, researchers at Concordia University's Institute of Information Systems Engineering are working toward a cure.

PhD candidate Ola Amayri and thesis supervisor, Nizar Bouguila, have conducted a comprehensive study of several in the process of developing a new and efficient one. They have now proposed a new statistical framework for spam filtering that quickly and efficiently blocks unwanted messages.

"The majority of previous research has focused on the textual content of spam emails, ignoring visual content found in , such as images. By considering patterns from text and images simultaneously, we've been able to propose a new method for filtering out spam," says Amayri, who recently published her findings online in a series of international conferences and peer-reviewed journals.

Amayri explains that new often employ sophisticated tricks, such as deliberately obscuring text, obfuscating words with symbols, and using batches of the same images with different backgrounds and colours that might contain random text from the web. However, until now, the majority of research in the domain of email spam filtering has focused on the automatic extraction and analysis of the textual content of spam emails and has ignored the rich nature of image-based content. When these tricks are used in combination, traditional spam filters are powerless to stop the messages, because they normally focus on either text or images but rarely both.

So how do we stop spam before it sullies our inboxes? "Our new method for spam filtering is able to adapt to the dynamic nature of spam emails and accurately handle spammers' tricks by carefully identifying informative patterns, which are automatically extracted from both text and images content of spam emails," says Amayri.

By conducting extensive experiments on traditional spam filtering methods that were general and limited to patterns found in texts or images, she has developed a much stronger way, based on techniques used in pattern recognition and data mining, to filter out unwanted emails. Although the new method has been tested on English spam emails, Amayri says it can be easily extended to other languages.

While this new spam-detecting approach is still in the development stage, Amayri and Bouguila are currently working on a plug-in for SpamAssassin, the world's most widely used open-source spam filter. Amayri hopes that this plug-in will allow other researchers to perform further tests and make more progress in the field of spam detection.

"Spammers keep adapting their methods so that they can trick the spam filters, says Amayri. "Researchers in this field need to band together to keep adapting our methods too, so that we can keep out and focus on those messages that are really important."

Explore further: Computer-assisted accelerator design

Related Stories

Image spam grows to 20 percent of spam

Jul 24, 2006

E-mail spam that relies on images to make it difficult to filter now encompasses 21 percent of all spam, said IronPort Systems, a San Bruno, Calif., company.

Revolutionary Spam Firewall

Aug 23, 2004

The email spam nightmare could be halted in cyberspace by a groundbreaking firewall developed at The University of Queensland. The new technology is the only true spam firewall in existence, according to co-developer Matthew Sulli ...

US senators seek to squash spam

Apr 03, 2009

Two US senators declared war on spam on Thursday. Senator Olympia Snowe, a Republican from Maine, and Senator Bill Nelson, a Democrat from Florida, introduced legislation aimed at curbing unsolicited text ...

Recommended for you

Computer-assisted accelerator design

Apr 22, 2014

Stephen Brooks uses his own custom software tool to fire electron beams into a virtual model of proposed accelerator designs for eRHIC. The goal: Keep the cost down and be sure the beams will circulate in ...

First steps towards "Experimental Literature 2.0"

Apr 21, 2014

As part of a student's thesis, the Laboratory of Digital Humanities at EPFL has developed an application that aims at rearranging literary works by changing their chapter order. "The human simulation" a saga ...

User comments : 6

Adjust slider to filter visible comments by rank

Display comments: newest first

Noumenon
2 / 5 (12) Nov 20, 2012
More proof that social engineering "progressive" liberals want government control of every aspect of your lives,

Democrats now desire warrantless access to Americans' e-mail by the federal government.

The same group of OWS type of Obama voters who complained about the Patriot Act, will accept this legislation without protest. The current generation is the weakest, dumbest, and useless in US history.
Urgelt
5 / 5 (3) Nov 20, 2012
Noumenon, when are you going to wake up?

It isn't 'progressive liberals' behind the rise of the surveillance state, it's 'authoritarians.'

Obama is one. So was Bush II. So was Clinton, and Bush I, and Reagan. Not one of those presidents was progressive, whatever Fox Noise lies you've heard. All contributed to our slide towards authoritarian government. None embodied progressive principles.

The last arguably 'progressive' president was LBJ, and he was more a centrist than a progressive. Obama falls somewhere to the right of Nixon on the political scale.

Nixon never championed reverse socialism on the scale Bush and Obama together pulled off. Trillions in bad debt moved to the taxpayer to benefit wealthy investors... that isn't progressive. Spying on Americans - not progressive. Assassinating people not on any battlefield, rather than arresting and prosecuting their asses - not progressive. Drug war - not progressive.

I don't think you know what progressive means.
Noumenon
2.3 / 5 (9) Nov 21, 2012
I think most Obama voters have little understanding of progressive liberalisms logical conclusion.

Taking government control of one-sixth of the US economy, in Obama care, is big gov liberal progressivism. The next logical step is to reduce health care costs by controlling human behavior ....

The liberal progressives are all about the expansion of government for social justice, by implementing redistribution of wealth and social engineering. They will use statisticians to study every aspect of society, and enforce regulations and laws that aim to control human behavior, for "our own good" or "the greater good".

See Cass Sunstein's book, "Nudge",... a Obama administration appointee. Or the UN's agenda 21.

As I said before , the greatest threat to personal liberty is the social progressive and their army of statisticians.
xeb
1 / 5 (1) Nov 27, 2012
[#1] Could someone tell me why there is no such system, that every email owner/provider sets local-context-dependent blockade for every incoming email. For example: a simple questions: "what city do i live" (is our company located at); (or even: what age and/or sex am i?"), "what is my proffesion" etc. .. would make impossible to easily (eg automaticly) send many spam emails. X writes an email, tries to send it to Y, and his box's outbound mechanism communicates "the receiving server demands completing this form" (otherwise denies to receive). There are countless possible context-dependent "filter forms" for this - e.g. a very strong ones like: "i accept email only from friend, please provide 40-char password i gave you (if you are one)". Or pictures: "please click one of these 10 face-shapes - the one that resembles email owner most".
xeb
1 / 5 (1) Nov 27, 2012
[#2] And more points:
- sender immediatelly knows about succes/failure
- there could be services allowing to bypass context-filter: [pay to send] or [case-check by human administrator]
- another example: in order to send an email one must provide information available on some www, but not easily. E.g. "when was our company established, how many people do we employ, do we operate in africa" etc for some companies/institutions .. Spamer would spend hours to send 20 emails
I think you got the idea - its clear. So, are there such systems in use? ... :)
(My guess: in fact email service providers do not want to decrease traffic - spaming probably does create earnings. And above idea is not to much restrictive - email owner/manager could easily choose more and more general "pass-questions").
(please excuse my language-use incompetence)
xeb
1 / 5 (1) Nov 27, 2012
[in short, partially stepping back:] ok, ive found it:
"Challenge-response_spam_filtering"
(...)"Advocates of C/R systems argue that the benefits by far outweigh the 'burden' of an incidental challenge and that there will probably never be a final solution against spam without laying some kind of burden on the e-mail sender. They reason that the more widespread the use of C/R systems will be, the more understood, accepted and appreciated they will be. In an analogy with snail mail, the sender is prepared to pay for the stamp," (...)

More news stories

US urged to drop India WTO case on solar

Environmentalists Wednesday urged the United States to drop plans to haul India to the WTO to open its solar market, saying the action would hurt the fight against climate change.

Is nuclear power the only way to avoid geoengineering?

"I think one can argue that if we were to follow a strong nuclear energy pathway—as well as doing everything else that we can—then we can solve the climate problem without doing geoengineering." So says Tom Wigley, one ...