Google team: Self-teaching computers recognize cats

June 26, 2012 by Nancy Owano, report
The architecture and parameters in one layer of the network. The overall network replicates this structure three times. For simplicity, the images are in 1D. Image from arXiv:1112.6209v3

( -- At the International Conference on Machine Learning, which starts today in Edinburgh, participants will hear about Google’s results after several years’ work at their big idea laboratory, Google X. Computers can teach themselves to recognize cats. The artificial neural network had successfully taught itself on its own to identify these animals. The team of scientists and programmers, in their paper titled “Building high-level features using large scale unsupervised learning,” describe how they trained computers on a dataset of 10 million images, each with 200x200 pixels.

In one of the largest neural networks for machine learning, the team connected 16,000 computer processors and used the pool of 10 million images, taken from YouTube videos. A brain-like neural network was then put to work. The research team was led by Stanford University computer scientist Andrew Y. Ng and Google fellow Jeff Dean. The “brain” assembled a dreamlike digital image of a cat by using a hierarchy of memory locations to cull features after exposure to the millions of images. Presented with the digital images, Google’s brain looked for cats.

In the human brain, as biologists suggest, neurons detect significant objects and that is what the software closely mirrored, described as turning out to be a “cybernetic cousin” to what takes place in the human visual cortex.

“We would like to understand if it is possible to learn a face detector using only unlabeled images downloaded from the Internet,” said the authors, describing the purpose at the outset of their research. “Contrary to what appears to be a widely held negative belief, our experimental results reveal that it is possible to achieve a face detector via only unlabeled data. Control experiments show that the feature detector is robust not only to translation but also to scaling and 3D rotation,” they said.

Their work in self-teaching machines is an example of scientific interest in what clusters of computers can achieve now in learning systems. According to Ng, the idea is that “You throw a ton of data at the algorithm and you let the data speak and have the software automatically learn from the data.”

At the same time, he is reluctant to even suggest that what scientists are achieving exactly mirrors the human brain, as computing capacity is still dwarfed by the number of connections in the brain. “A loose and frankly awful analogy is that our numerical parameters correspond to synapses,” he said.

Explore further: Mapping the brain

More information:

Building high-level features using large scale unsupervised learning, arXiv:1112.6209v3 [cs.LG]

We consider the problem of building high- level, class-specific feature detectors from only unlabeled data. For example, is it possible to learn a face detector using only unlabeled images? To answer this, we train a 9-layered locally connected sparse autoencoder with pooling and local contrast normalization on a large dataset of images (the model has 1 bil- lion connections, the dataset has 10 million 200x200 pixel images downloaded from the Internet). We train this network using model parallelism and asynchronous SGD on a clus- ter with 1,000 machines (16,000 cores) for three days. Contrary to what appears to be a widely-held intuition, our experimental re- sults reveal that it is possible to train a face detector without having to label images as containing a face or not. Control experiments show that this feature detector is robust not only to translation but also to scaling and out-of-plane rotation. We also find that the same network is sensitive to other high-level concepts such as cat faces and human bod- ies. Starting with these learned features, we trained our network to obtain 15.8% accu- racy in recognizing 20,000 object categories from ImageNet, a leap of 70% relative im- provement over the previous state-of-the-art.

Related Stories

Mapping the brain

August 26, 2011

The brain of a mouse measures only 1 cubic centimeter in volume. But when neuroscientists at Harvard’s Center for Brain Science slice it thinly and take high-resolution micrographs of each slice, that tiny brain turns ...

Image processing: The human (still) beats the machine

October 31, 2011

( -- A novel experiment conducted by researchers at Idiap Research Institute and Johns Hopkins University highlights some of the limitations of automatic image analysis systems. Their results were recently published ...

Teaching machines to recognize shapes

December 12, 2011

As any parent knows, teaching a toddler to recognize objects involves trial-and-error. A child, for example, may not initially recognize a cow in a picture-book after seeing the live animal on a farm and being told its label. ...

Madrid duo fire up quantum contender to Google search

December 14, 2011

( -- Two Madrid scientists from The Complutense University think they have an algorithm that may impact the nature of the world's leading search engine. In essence, they are saying Hey, world, Google This. "We ...

When robots learn from our mistakes

May 26, 2011

( -- Robots typically acquire new capacities by imitation. Now, EPFL scientists are doing the inverse -- developing machines that can learn more rapidly and outperform humans by starting from failed or inaccurate ...

Understanding the propagation of Alzheimer's disease

March 19, 2012

(Medical Xpress) -- The connections between neurons might play a role in neurodegenerative diseases, including Alzheimer’s Disease (AD). In a pioneering approach to studying how neurodegenerative diseases like AD spread ...

Recommended for you

Robot designed for faster, safer uranium plant pipe cleanup

April 21, 2018

Ohio crews cleaning up a massive former Cold War-era uranium enrichment plant in Ohio plan this summer to deploy a high-tech helper: an autonomous, radiation-measuring robot that will roll through miles of large overhead ...

After Facebook scrutiny, is Google next?

April 21, 2018

Facebook has taken the lion's share of scrutiny from Congress and the media about data-handling practices that allow savvy marketers and political agents to target specific audiences, but it's far from alone. YouTube, Google ...

How social networking sites may discriminate against women

April 20, 2018

Social media and the sharing economy have created new opportunities by leveraging online networks to build trust and remove marketplace barriers. But a growing body of research suggests that old gender and racial biases persist, ...

Virtually modelling the human brain in a computer

April 19, 2018

Neurons that remain active even after the triggering stimulus has been silenced form the basis of short-term memory. The brain uses rhythmically active neurons to combine larger groups of neurons into functional units. Until ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

5 / 5 (4) Jun 26, 2012
Awesome. Next the internet will be able to generate its own cat memes. I don't think this is how the robo-apocalypse is supposed to go down.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.