The next big breakthrough in robotics

January 24, 2018 by Bill Ibelle And Allie Nicodemo, Northeastern University
Credit: Northeastern University

While drones and driverless cars dominate the headlines, another breakthrough—robot dexterity—is likely to have an even greater impact in both business and everyday life.

"Robot manipulation is the next shoe to drop," says Robert Platt, computer science professor and head of the Helping Hands robotics lab at Northeastern. "Imagine a robot that can do things with it's hands in the real world—anything from defusing a bomb to doing your laundry. This has been a dream in the research community for decades, but now we're finally getting to the point where it could actually happen."

Recent advances in machine learning, Big Data, and robot perception have put us on the threshold of a quantum leap in the ability of robots to perform fine motor tasks and function in uncontrolled environments, says Platt.

It's the difference between robots that can do repetitive tasks in a highly-structured factory environment and a new era of humanoid robots that can do meaningful work in the real world.

Why fine motor skills have lagged

There's an irony in the field of robots and artificial intelligence known as Moravek's Paradox: What's hard for humans is relatively easy for robots, and what's easy for humans is nearly impossible for robots.

We can program a robot with the computational skills to defeat an international chess champion, but struggle to give it the dexterity of a 2-year-old. Identifying and grabbing a pencil in a random pile of office supplies is nearly impossible for a robot, and opening a door and walking into a room—as demonstrated in this recent international robot competition—can look like a scene from a slapstick comedy.

Because humans have been evolving their visual, sensory, and motor skills for millions of years, these complex skills are so deeply ingrained in human circuitry that we perform them unconsciously. In contrast, high-level endeavors such as mathematics, science, and stock analysis are relatively recent human endeavors—so they're much easier for engineers to replicate.

Poised for a breakthrough

In spite of the enormous challenge, Platt says autonomous robots are poised to make huge leaps in their ability to manipulate unfamiliar objects.

For example, Platt and his team at the Helping Hands Lab have trained a robot to find, grab, and remove unfamiliar objects from a pile of clutter with 93 percent accuracy. Achieving this required significant advances in , perception, and control.

The researchers used a technique called reinforcement learning in which the robot learns via trial and error. They created a simulated world in which the robot could practice picking up and manipulating objects in virtual reality. When the robot did what the researchers wanted—grabbed an object from a pile—it was given a reward. This technique allows the robot to master skills in a virtual environment and then apply them to the real world.

A major advance in depth perception was also essential for robots to work in an uncontrolled environment. Previously, they could only see the world as a flat field of seemingly random colors. But with this new 3-D perception, they could identify individual objects in a crowded field.

While vision is an excellent tool for guiding broad movements, require a sense of touch.

"Think of what you can do with gloves on," explains Platt. "You can open the garage door, grab a shovel, and clear the driveway. But if you need to unlock the garage first, you need to take your gloves off to insert the key."

As part of a NASA grant, Platt's lab recently built a robotic hand equipped with tactile sensors and developed new algorithms for interpreting the tactile data.

"In order to insert a key into a lock, the robot needs to know exactly how it's holding the key, down to the millimeter," says Platt. "Our algorithms can localize these kinds of grapsed objects very accurately."

Credit: Northeastern University

Platt's lab demonstrated these new capabilities by grasping a USB connector and plugging it into a port. While this may not sound like a big deal, it's a critical step toward creating robots that can do precise manipulation tasks such as changing the battery in a cell phone.

What's coming?

As with any fledgling advancement—radar, the telephone, the internet—the practical applications of robot dexterity are hard to predict. But here are a few:


Platt's Helping Hands lab—in collaboration with the University of Massachusetts Lowell and the Crotched Mountain Rehabilitation Facility in New Hampshire—is building a power wheelchair with a robotic arm that can grasp items around the house or perform simple household tasks. This could enable elderly or people with disabilities to continue to live independently in their homes.

Platt is also interested in adapting this technology for everyday use. "We hear a lot about the Alexa-style assistants that can answer questions by accessing the internet. But these assistants can't do anything physical," says Platt. "We want to equip these devices with a robotic body so you can say, 'Alexa, get the newspaper,' or 'Alexa, clean up Jimmy's room.'"


Engineering professor Hanumant Singh in collaboration with Platt are building a golf-cart sized mobile robot equipped with a robotic arm that can drive around Northeastern's campus autonomously and perform simple manipulation tasks like taking out the trash.


Similar types of robots could take on similar duties in areas of intense conflict and be used for dangerous operations such as defusing mines. For example, Platt and his group recently completed a grant from the Office of Naval Research to develop fundamental manipulation technologies that will be used aboard Naval vessels.

Hazardous waste

Engineering professor Taskin Padir and his team received a grant from the Department of Energy to adapt NASA's Valkyrie robot for hazardous waste disposal. There are more than a dozen sites scattered around the U.S. where radioactive waste was buried in tunnels during the Cold War. For to locate, grasp, and place this waste in safe containers, they will need fine motor skills and an ability to operate in unfamiliar environments.


Funded by a grant from the National Science Foundation, engineering professor Peter Whitney is working with researchers at Stanford University to create a that can perform MRI-guided surgery.

Space exploration

Platt is working with researchers at NASA to develop robotic manipulation capabilities for handling soft objects on future NASA space missions.

"Robots that work flawlessly in the lab break down quickly when they're placed in unfamiliar situations," says Platt. "Our goal is to develop the underlying algorithms that will allow them to be more reliable in the real world. Ultimately, this will fundamentally change the way we think about robots, allowing them to become partners with humans rather than just machines that work in far away factories."

Explore further: New robots can see into their future

Related Stories

New robots can see into their future

December 4, 2017

University of California, Berkeley, researchers have developed a robotic learning technology that enables robots to imagine the future of their actions so they can figure out how to manipulate objects they have never encountered ...

Researchers study human movement to build better robots

November 7, 2017

Draw a figure eight in the air. It might feel like one swift movement. But in fact, the velocity of your hand and arm likely varies, traveling faster through the straight parts and slowing down during the curves.

NASA's Valkyrie robots set the table for human life on Mars

May 19, 2016

Four sister robots built by NASA could be pioneers in the colonization of Mars, part of an advance construction team that sets up a habitat for more fragile human explorers. But first they're finding new homes on Earth and ...

Software enables robots to be controlled in virtual reality

December 14, 2017

Even as autonomous robots get better at doing things on their own, there will still be plenty of circumstances where humans might need to step in and take control. New software developed by Brown University computer scientists ...

Recommended for you

Sculpting stable structures in pure liquids

February 21, 2019

Oscillating flow and light pulses can be used to create reconfigurable architecture in liquid crystals. Materials scientists can carefully engineer concerted microfluidic flows and localized optothermal fields to achieve ...

Researchers make coldest quantum gas of molecules

February 21, 2019

JILA researchers have made a long-lived, record-cold gas of molecules that follow the wave patterns of quantum mechanics instead of the strictly particle nature of ordinary classical physics. The creation of this gas boosts ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.