Scientists make a maze-running artificial intelligence program that learns to take shortcuts

May 11, 2018 by Amina Khan, Los Angeles Times
An AI system learns to take shortcuts. Credit: Nature (2018). DOI: 10.1038/s41586-018-0102-6

Call it an a-MAZE-ing development: A U.K.-based team of researchers has developed an artificial intelligence program that can learn to take shortcuts through a labyrinth to reach its goal. In the process, the program developed structures akin to those in the human brain.

The emergence of these computational "," described in the journal Nature, could help scientists design better navigational software for future robots and even offer a new window through which to probe the mysteries of the mammalian brain.

In recent years, AI researchers have developed and fine-tuned deep-learning networks—layered programs that can come up with novel solutions to achieve their assigned goal. For example, a deep-learning network can be told which face to identify in a series of different photos, and through several rounds of training, can tune its algorithms until it spots the right face virtually every time.

These networks are inspired by the brain, but they don't work quite like them, said Francesco Savelli, a neuroscientist at Johns Hopkins University who was not involved in the paper. So far, AI systems don't come close to emulating the brain's architecture, the diversity of real neurons, the complexity of individual neurons or even the rules by which they learn.

"Most of the learning is thought to occur with the strengthening and weakening of these synapses," Savelli said in an interview, referring to the connections between neurons. "And that's true of these AI systems too—but exactly how you do it, and the rules that govern that kind of learning, might be very different in the brain and in these systems."

Regardless, AI has been really useful for a number of functions, from facial recognition to deciphering handwriting and translating languages, Savelli said. But higher-level activities—such as navigating a complex environment—have proved far more challenging.

One aspect of navigation that our brains seem to perform without conscious effort is path integration. Mammals use this process to recalculate their position after every step they take by accounting for the distance they've traveled and the direction they're facing. It's thought to be key to the brain's ability to produce a map of its surroundings.

Interview with Caswell Barry about grid cells. Credit: DeepMind

Among the neurons associated with these "cognitive maps": , which light up when their owner is in some particular spot in the environment; head-direction , which tell their owner what direction they're facing; and grid cells, which appear to respond to an imaginary hexagonal grid mapped over the surrounding terrain. Every time a person steps on a "node" in this grid, the neuron fires.

"Grid cells are thought to endow the cognitive map with geometric properties that help in planning and following trajectories," Savelli and fellow Johns Hopkins neuroscientist James Knierim wrote in a commentary on the paper. The discovery of grid cells earned three scientists the 2014 Nobel Prize in physiology or medicine.

Humans and other animals seem to have very little trouble moving through space because all of these highly specialized neurons work together to tell us where we are and where we're going.

Scientists at DeepMind, which is owned by Google and University College London, wondered whether they could develop a program that could also perform path integration. So they trained the network with simulations of paths used by rodents looking for food. They also gave it data for a rodent's movement and speed as well as feedback from simulated place cells and head-direction cells.

During this training, the researchers noticed something strange: The simulated rodent appeared to develop patterns of activity that looked remarkably like grid cells—even though grid cells had not been part of their training system.

"The emergence of grid-like units is an impressive example of deep learning doing what it does best: inventing an original, often unpredicted internal representation to help solve a task," Savelli and Knierim wrote.

Interview with Matt Botvinick about neuroscience and AI. Credit: DeepMind

Grid cells appear to be so useful for path integration that this faux-rodent came up with a solution eerily similar to a real rodent brain. The researchers then wondered: Could grid cells also be useful in another crucial aspect of mammal navigation?

That aspect, called vector-based navigation, is basically the ability to calculate the straight-shot, "as the crow flies" distance to a goal even if you originally took a longer, less-direct route. That's a useful skill for finding shortcuts to your destination, Savelli pointed out.

To test this, researchers challenged the grid-cell-enabled faux-rodent to solve a maze, but blocked off most of the doorways so the program would have to take the long route to its goal. They also modified the program so it was rewarded for actions that brought it closer to the goal. They trained the network on a given maze and then opened shortcuts to see what happened.

Sure enough, the simulated rodent with grid cells quickly found and used the shortcuts, even though those pathways were new and unknown. And it performed far better than a faux-rodent whose start point and goal point were tracked only by place cells and head-direction cells. It even beat out a "human expert," the study authors said.

The findings eventually could prove useful for robots making their way through unknown territory, Savelli said. And from a neuroscientific perspective, they could help researchers better understand how these neurons do their job in the mammalian brain.

Of course, this program was highly simplified compared to its biological counterpart, Savelli pointed out. In the simulated rodent, the "place cells" didn't change—even though place cells and grid cells influence each other in complex ways in real brains.

"By developing the network such that the place-cell layer can be modulated by -like inputs, we could begin to unpack this relationship," Savelli and Knierim wrote.

Developing this AI program further could help scientists start to understand all the complex relationships that come into play in living neural systems, they added.

But whether they want to hone the technology or use it to understand biology, scientists will have to get a better handle on their own deep-learning programs, whose solutions to problems are often hard to decipher even if they consistently get results, scientists said.

"Making deep-learning systems more intelligible to human reasoning is an exciting challenge for the future," Savelli and Knierim wrote.

Explore further: Self-navigating AI learns to take shortcuts: study

More information: Andrea Banino et al. Vector-based navigation using grid-like representations in artificial agents, Nature (2018). DOI: 10.1038/s41586-018-0102-6

Related Stories

New model for the origin of grid cells

July 21, 2017

Ludwig Maximilian University of Munich neurobiologists present a new theory for the origin of the grid cells required for spatial orientation in the mammalian brain, which assigns a vital role to the timing of trains of signals ...

Better understanding of mapmaking in the brain

August 9, 2010

"Grid cells," which help the brain map locations, have been found for the first time outside of the hippocampus in the rat brain, according to new research from the Norwegian University of Science and Technology (NTNU). The ...

Recommended for you

Robots as tools and partners in rehabilitation

August 17, 2018

In future decades, the need for effective strategies for medical rehabilitation will increase significantly, because patients' rate of survival after diseases with severe functional deficits, such as a stroke, will increase. ...

Security gaps identified in internet protocol IPsec

August 15, 2018

In collaboration with colleagues from Opole University in Poland, researchers at Horst Görtz Institute for IT Security (HGI) at Ruhr-Universität Bochum (RUB) have demonstrated that the internet protocol IPsec is vulnerable ...

Researchers find flaw in WhatsApp

August 8, 2018

Researchers at Israeli cybersecurity firm said Wednesday they had found a flaw in WhatsApp that could allow hackers to modify and send fake messages in the popular social messaging app.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.