Steering material scientists to better memory devices

October 11, 2018 by Stefano Ambrogio, IBM
Steering material scientists to better memory devices
Graphical representation of a crossbar array, where different memory devices serve in different roles. Credit: IBM

Ideally, next-generation AI technologies should understand all our requests and commands, extracting them from a huge background of irrelevant information, in order to rapidly provide relevant answers and solutions to our everyday needs. Making these "smart" AI technologies pervasive—in our smartphones, our homes, and our cars—will require energy-efficient AI hardware, which we at IBM Research plan to build around novel and highly capable analog memory devices.

In a recent paper published in Journal of Applied Physics, our IBM Research AI team established a detailed set of guidelines that emerging nano-scaled analog memory devices will need to satisfy in order to enable such energy-efficient AI hardware accelerators.

We had previously shown, in a Nature paper published in June 2018, that training a neural network using highly parallel computation within dense arrays of memory devices such as phase-change memory is faster and consumes less power than using a graphics processing unit (GPU).

The advantage of our approach comes from implementing each weight with multiple devices, each serving in a different role. Some devices are mainly tasked with memorizing long-term information. Other devices are updated very rapidly, changing as training images (such as pictures of trees, cats, ships, etc.) are shown, and then occasionally transferring their learning to the long-term information devices. Although we introduced this concept in our Nature paper using existing devices (phase change memory and conventional capacitors), we felt there should be an opportunity for new memory devices to perform even better, if we could just identify the requirements for these devices.

In our follow-up paper, just published in Journal of Applied Physics, we were able to quantify the device properties that these "long-term information" and "fast-update" devices would need to exhibit. Because our scheme divides tasks across the two categories of devices, these requirements are much less stringent—and thus much more achievable—than before. Our work provides a clear path for material scientists to develop novel devices for energy-efficient AI hardware accelerators based on analog .

Explore further: A new brain-inspired architecture could improve how computers handle data and advance AI

More information: Giorgio Cristiano et al. Perspective on training fully connected networks with resistive memories: Device requirements for multiple conductances of varying significance, Journal of Applied Physics (2018). DOI: 10.1063/1.5042462

Stefano Ambrogio et al. Equivalent-accuracy accelerated neural-network training using analogue memory, Nature (2018). DOI: 10.1038/s41586-018-0180-5

Related Stories

Capacitor-based architecture for AI hardware accelerators

July 10, 2018

IBM is reaching beyond digital technologies with a capacitor-based cross-point array for analog neural networks, exhibiting potential orders of magnitude improvements in deep learning computations. Analog computing architectures ...

The future of hardware is AI

December 7, 2017

AI workloads are different from the calculations most of our current computers are built to perform. AI implies prediction, inference, intuition. But the most creative machine learning algorithms are hamstrung by machines ...

Novel synaptic architecture for brain inspired computing

July 9, 2018

The brain and all its magnificent capabilities is powered by less than 20 watts. Stop to think about that for a second. As I write this blog my laptop is using about 80 watts, yet at only a fourth of the power, our brain ...

Recommended for you

Physicist describes the shape of a wormhole

October 17, 2018

A RUDN physicist demonstrated how to describe the shape of any symmetrical wormhole—a black hole that theoretically can be a kind of a portal between any two points in space and time—based on its wave spectrum. The research ...

Physics: Not everything is where it seems to be

October 16, 2018

Scientists at TU Wien, the University of Innsbruck and the ÖAW have for the first time demonstrated a wave effect that can lead to measurement errors in the optical position estimation of objects. The work now published ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.