New mathematical model to enable web searches for meaning

In a paper to be published online in Computational Linguistics tomorrow, Dr. Daoud Clarke describes how he has built a mathematical model based on the idea that the meaning of words and phrases is determined by the contexts in which they occur.

“This is an old idea, with its origin in the philosophy of Wittgenstein, and was later taken up by linguists,” said Dr. Clarke, “but this is the first time that someone has used it to construct a comprehensive theory of meaning.”

The model provides a theory about how to represent words and phrases using vectors, or sequences of numbers. A vector identifies a point in some multi-dimensional space, which may have hundreds or thousands of dimensions.

“There are existing techniques which can build these vectors for words by looking at the contexts they occur in, for example on the web. This works well for words or short phrases, but if you want to extend this to long phrases or whole sentences, you quickly run out of data, even on the web. Our theory tells you what the vector for a phrase should look like in terms of the vectors for the individual words that make up the phrase.

“For example, at the moment we may have vectors for 'big' and 'cat', but we don't know the best way to combine them to get a vector for 'big cat',” explained Dr. Clarke. “There are lots of possibilities: for example you could add the two vectors together, but then 'big cat' would have to mean the same as 'cat big', which doesn't make sense.

The value of the theory is in identifying which methods of combining vectors do make sense. Our theory will tell you if your method of combining vectors is consistent with the idea that meaning is determined by context.”

According to Dr. Clarke, most existing theories in this field are based around the idea that the meaning of sentences can be represented in terms of logic, but these cannot capture the subtleties of language, such as the relationship between the words “like” and “love”. Representing meanings of words using vectors allows fuzzy relationships between words to be expressed as the distance or angle between the vectors.

Dr. Clarke believes his theory will have many applications in artificial intelligence, in particular helping web search engines understand the meaning of your query. “Google works by looking for the words you type in the documents it knows about,” said Dr Clarke. “If you type in a long phrase or sentence, it just tries to match as many words as possible. Imagine how powerful it could be if it understood the meaning of your query, and tried to match it to the closest meaning in all the documents it knows about.”

More information: A Context-theoretic Framework for Compositionality in Distributional Semantics, Computational Linguistics.

Provided by University of Herfordshire