Can networked human computation solve computer language comprehension?

Jan 26, 2009

Researchers at the University of Essex hope to answer this question by getting more volunteers to take part in their online game, Phrase Detectives.

Jon Chamberlain, from Essex's School of Computer Science and Electronic Engineering, explains: ‘Human language is not an unconnected series of words, phrases and sentences but a series of people, objects and ideas that refer to each other in different ways. The complexity of language makes it sound "natural" to a reader but it can be difficult to define the rules that allow us to understand it.

‘Consider the statement: "Mary is a teacher who is 25 years old. She lives in England." A human reader can easily ascertain facts about Mary's occupation, age and residence by, for example, knowing that the word "she" refers to the person "Mary". However, comprehending this type of language referencing is a challenge facing programmers when designing computer systems that try to understand text, such as search, translation and summarisation systems.’

This is where the work of those playing Phrase Detectives becomes important. The game, part of a larger project called AnaWiki, is an attempt to address the bottleneck in creating annotated linguistic resources. By initially investigating anaphoric references (as in the example above) the project aims to develop a resource larger than anything currently available.

Players (or detectives) register at: www.phrasedetectives.org and read through texts, making annotations to highlight relationships between words and phrases. They may be asked to 'name the culprit', so will be given a word or phrase and must look for it appearing earlier in the text. For example: 'Sherlink Holmes went to the shop. He got some tobacco for his pipe.' The word ‘he’ refers to 'Sherlink Holmes'.

Jon added: ‘Players of the game are helping to create a resource that is rich in linguistic information and improves future technology. This project aims to collect a significant amount of data and investigate the possibility of using mass collaboration to train computer systems.

‘The best way to understand a language is to have lots of examples where the meaning has been clarified. Unfortunately creating this type is resource is both time consuming and expensive but the new approach offered by Phrase Detective should address this resource shortage. The same methodology could also be used to create resources for machine translation, semantics and other linguistic phenomenon.’

So far, players have made over 40,000 annotations in four weeks. However, the researchers hope more will join as detectives and that people will add new text to the site for analysis.

Phrase Detectives can be defined as part of a genre of “games with a purpose” (GWAP) that collect data on images, texts and music. The crucial element of these games is that players receive points for agreeing with each other. They are motivated to collaborate with their partners in order to score maximum points. This ensures that players are attempting to provide good quality information, as this will result in the most agreement.

The Essex researchers believe Phrase Detectives is the first attempt to collect linguistic judgements using a fun, collaborative online game. They aim to make the tasks and the texts interesting so it feels more like a computer game than a linguistic task. The data collected can then be used to improve computer systems that try to understand text. For example, it could help search engines find information more relevant to your searches.

So, can networked human computation really solve complex language comprehension tasks on computers? Initial results from the beta version of the game look promising and more detailed analysis will completed in early 2009.

Source: University of Essex

Explore further: MU researchers develop more accurate Twitter analysis tools

add to favorites email to friend print save as pdf

Related Stories

Cryptozoology? No need for an apology

Mar 15, 2013

All forms of science are reliant on facts, hard evidence and statistics to maintain relevance and credibility. But what of the legitimacy of the so-called "pseudosciences"?

Apple kicks SMS scam fraudsters to the curb

Dec 14, 2012

(Phys.org)—Just what you never wanted. Mac-based malware, just ponder that phrase alone, not Windows-based but Mac-based, that tricks users into paying subscription fees. The malware masquerades as an installer ...

Computer scientist seeks the real meaning of language

Oct 27, 2011

Better be careful about telling a lie to Julia Hirschberg. The computer science professor, an expert in spoken language, examines what people unconsciously communicate through such things as intonation, accent ...

Recommended for you

Avatars make the Internet sign to deaf people

6 hours ago

It is challenging for deaf people to learn a sound-based language, since they are physically not able to hear those sounds. Hence, most of them struggle with written language as well as with text reading ...

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

gmurphy
not rated yet Jan 26, 2009
wow, this is really cool, a solid dataset for training text comprehension algorithms.