Novel synaptic architecture for brain inspired computing

July 9, 2018 by Irem Boybat, IBM
Credit: CC0 Public Domain

The brain and all its magnificent capabilities is powered by less than 20 watts. Stop to think about that for a second. As I write this blog my laptop is using about 80 watts, yet at only a fourth of the power, our brain outperforms state-of-the-art supercomputers by several orders of magnitude when it comes to energy efficiency and volume. Nature is truly remarkable.

For this reason it shouldn't be surprising that scientists around the world are seeking inspiration from the human brain as a promising avenue towards the development of next generation AI computing systems and while the IT industry has made significant progress in the past several years, particularly in using machine learning for computer vision and speech recognition, current technology is hitting a wall when it comes to matching the power efficiency of their biological counterpart, but this could be about to change.

As reported last week in Nature Communications, my colleagues and I at IBM Research and collaborators at EPFL and the New Jersey Institute of Technology have developed and experimentally tested an artificial synapse architecture using 1 million devices—a significant step towards realizing large-scale and energy efficient neuromorphic computing technology.

When the brilliant scientist John von Neumann built today's computer architecture, which powers nearly 100 percent of the world's computers, he kept the memory and the processing separately. This means data needs to constantly shuttle back and forth, generating heat and requiring a lot of energy – it's an efficiency bottleneck. The brain of course doesn't have different compartments, which is why it's so efficient. But this didn't deter teams from sticking with von Neumann's design to build a neural and while they have some success, the efficiency of these systems remains low – you simply can't beat nature.

More recently, scientists, including those at IBM, have taken a different approach based on a new class of nanoscale devices known as memristive devices, which have shown great promise in addressing this bottleneck. Our design is based on something called phase change memory (PCM), arguably the most advanced emerging non-volatile memory technology. An electric pulse is applied to the material, which changes the conductance of the device though its physical properties.

As explained in our paper: "Memristive devices such as PCM devices store information in their resistance/conductance states and exhibit conductivity modulation based on the programming history. The central idea in building cognitive hardware based on memristive devices is to store the synaptic weights as their conductance states and to perform the associated computational tasks in place. However, precise modulation of the device conductance over a wide dynamic range, necessary to maintain high network accuracy, is proving to be challenging."

Our breakthrough is in our design, which we call a multi-memristive synaptic architecture. This architecture enables us to increase the synaptic precision without increasing the power density even though we use several memristive devices to represent one synapse. The trick is that we have a nice selection mechanism, based on a global counter, which tells the device that it needs to change and when. The only penalty or cost is the requirement for more real-estate for the additional PCM devices.

To test our architecture, we trained both spiking and a non-spiking neural networks. Our data selected is a popular one – the MNIST data set of handwritten digits and our task is handwritten digit recognition – essentially our network needs to recognize what number is appearing from the handwritten images. In both cases, we see that the multi-memristive synapse significantly outperforms the conventional differential architectures with two devices, clearly illustrating the effectiveness of the proposed architecture. A highlight of the work is an experimental demonstration of the multi-memristive synaptic architecture in a spiking neural network using more than 1 million phase-change memory devices.

The architecture is applicable to a wide range of neural networks and memristive technologies and is crossbar-compatible. The proposed and its experimental demonstration are a significant step towards the realization of highly efficient, large-scale based on memristive devices with typical, experimentally observed non-ideal characteristics. Having said that, we are also focused on improving the memristive device itself, it's accuracy and dynamic range and then we think we can aim for the holy grail – floating point performance.

Explore further: Low-power, flexible memristor circuit for mobile and wearable devices

More information: Irem Boybat et al. Neuromorphic computing with multi-memristive synapses, Nature Communications (2018). DOI: 10.1038/s41467-018-04933-y

Related Stories

Neuromorphic computing mimics important brain feature

August 18, 2016

(—When you hear a sound, only some of the neurons in the auditory cortex of your brain are activated. This is because every auditory neuron is tuned to a certain range of sound, so that each neuron is more sensitive ...

Creating new devices that emulate human biological synapses

September 29, 2016

Engineers at the University of Massachusetts Amherst are leading a research team that is developing a new type of nanodevice for computer microprocessors that can mimic the functioning of a biological synapse—the place ...

Using pectin to advance neuron-like electronic systems

November 11, 2016

Most of us know pectin as a key ingredient for making delicious jellies and jams, not as a component for a complex hybrid device that links biological and electronic systems. But a team of Italian scientists have built on ...

Recommended for you

Not all stem cells are created equal, study reveals

March 22, 2019

Researchers from the University of Toronto's Institute for Biomaterials and Biomedical Engineering (IBBME) and the Donnelly Centre have discovered a population of cells – dubbed to be "elite" – that play a key role in ...

Ancient birds out of the egg running

March 22, 2019

The ~125 million-year-old Early Cretaceous fossil beds of Los Hoyas, Spain, have long been known for producing thousands of petrified fish and reptiles (Fig. 1). However, researchers have uncovered an extremely rare, nearly ...

Making solar cells is like buttering bread

March 22, 2019

Formamidinium lead iodide is a very good material for photovoltaic cells, but getting the correct stable crystal structure is a challenge. The techniques developed so far have produced poor results. However, University of ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.