Can networked human computation solve computer language comprehension?

Jan 26, 2009

Researchers at the University of Essex hope to answer this question by getting more volunteers to take part in their online game, Phrase Detectives.

Jon Chamberlain, from Essex's School of Computer Science and Electronic Engineering, explains: ‘Human language is not an unconnected series of words, phrases and sentences but a series of people, objects and ideas that refer to each other in different ways. The complexity of language makes it sound "natural" to a reader but it can be difficult to define the rules that allow us to understand it.

‘Consider the statement: "Mary is a teacher who is 25 years old. She lives in England." A human reader can easily ascertain facts about Mary's occupation, age and residence by, for example, knowing that the word "she" refers to the person "Mary". However, comprehending this type of language referencing is a challenge facing programmers when designing computer systems that try to understand text, such as search, translation and summarisation systems.’

This is where the work of those playing Phrase Detectives becomes important. The game, part of a larger project called AnaWiki, is an attempt to address the bottleneck in creating annotated linguistic resources. By initially investigating anaphoric references (as in the example above) the project aims to develop a resource larger than anything currently available.

Players (or detectives) register at: www.phrasedetectives.org and read through texts, making annotations to highlight relationships between words and phrases. They may be asked to 'name the culprit', so will be given a word or phrase and must look for it appearing earlier in the text. For example: 'Sherlink Holmes went to the shop. He got some tobacco for his pipe.' The word ‘he’ refers to 'Sherlink Holmes'.

Jon added: ‘Players of the game are helping to create a resource that is rich in linguistic information and improves future technology. This project aims to collect a significant amount of data and investigate the possibility of using mass collaboration to train computer systems.

‘The best way to understand a language is to have lots of examples where the meaning has been clarified. Unfortunately creating this type is resource is both time consuming and expensive but the new approach offered by Phrase Detective should address this resource shortage. The same methodology could also be used to create resources for machine translation, semantics and other linguistic phenomenon.’

So far, players have made over 40,000 annotations in four weeks. However, the researchers hope more will join as detectives and that people will add new text to the site for analysis.

Phrase Detectives can be defined as part of a genre of “games with a purpose” (GWAP) that collect data on images, texts and music. The crucial element of these games is that players receive points for agreeing with each other. They are motivated to collaborate with their partners in order to score maximum points. This ensures that players are attempting to provide good quality information, as this will result in the most agreement.

The Essex researchers believe Phrase Detectives is the first attempt to collect linguistic judgements using a fun, collaborative online game. They aim to make the tasks and the texts interesting so it feels more like a computer game than a linguistic task. The data collected can then be used to improve computer systems that try to understand text. For example, it could help search engines find information more relevant to your searches.

So, can networked human computation really solve complex language comprehension tasks on computers? Initial results from the beta version of the game look promising and more detailed analysis will completed in early 2009.

Source: University of Essex

Explore further: Modeling the ripples of health care information

add to favorites email to friend print save as pdf

Related Stories

Cryptozoology? No need for an apology

Mar 15, 2013

All forms of science are reliant on facts, hard evidence and statistics to maintain relevance and credibility. But what of the legitimacy of the so-called "pseudosciences"?

Apple kicks SMS scam fraudsters to the curb

Dec 14, 2012

(Phys.org)—Just what you never wanted. Mac-based malware, just ponder that phrase alone, not Windows-based but Mac-based, that tricks users into paying subscription fees. The malware masquerades as an installer ...

Computer scientist seeks the real meaning of language

Oct 27, 2011

Better be careful about telling a lie to Julia Hirschberg. The computer science professor, an expert in spoken language, examines what people unconsciously communicate through such things as intonation, accent ...

Recommended for you

Forging a photo is easy, but how do you spot a fake?

Nov 21, 2014

Faking photographs is not a new phenomenon. The Cottingley Fairies seemed convincing to some in 1917, just as the images recently broadcast on Russian television, purporting to be satellite images showin ...

Algorithm, not live committee, performs author ranking

Nov 21, 2014

Thousands of authors' works enter the public domain each year, but only a small number of them end up being widely available. So how to choose the ones taking center-stage? And how well can a machine-learning ...

Professor proposes alternative to 'Turing Test'

Nov 19, 2014

(Phys.org) —A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally ...

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

gmurphy
not rated yet Jan 26, 2009
wow, this is really cool, a solid dataset for training text comprehension algorithms.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.