Communicating with drones using facial expressions and gestures could help save lives

July 25, 2018, Simon Fraser University
Computing science professor Richard Vaughan (right) and Ph.D. student Sepehr MohaimenianPour (left) are developing technologies in SFU's Autonomy Lab to help users interact with drones in a more intuitive manner. Credit: Simon Fraser University

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.

In Simon Fraser University's Autonomy Lab, researchers are using artificial intelligence to develop technologies that can make the more intuitive, without requiring the standard controller.

"Most commercial drones today come with controllers which work really well, but sometimes you may find yourself in a situation where your hands are busy," notes computing science professor Richard Vaughan, who leads this research.

"Or maybe you weren't expecting to interact with a drone today so you don't have special equipment with you. We'd like to be able to command drones in these situations and make the interaction natural and intuitive."

Vaughan's team is building a drone whose flight direction can be controlled by arm gestures—you can even command the drone to do a flip in mid-air.

Another drone the team has developed is one that executes commands based on the user's facial expressions. By recognizing a "trigger-face" the drone could be used to carry out actions such as taking a photo or video of an inaccessible area to assess a situation.

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.In Simon Fraser University's Autonomy Lab, researchers are using artificial intelligence to develop drone technologies that can make the human-robot interaction more intuitive, without requiring the standard controller. Credit: Simon Fraser University
"We would like to get to the point where interacting with a is as easy as working with a co-worker or a trained animal," says Vaughan.

As robot technology continues to advance, it is likely that one day we will be interacting with robots in our everyday life. Being able to communicate and provide commands to them without a controller is not only convenient but can help navigate emergency situations.

Fast Facts:

  • Fire Fighters in the U.S. are already using drones for emergency scenarios like fires and search and rescue missions.
  • Drones can be deployed very quickly. What used to take 10 or 15 minutes with just humans will only take 2-3 minutes with drones.
  • Currently, search and rescue teams send out 100 people out to walk on a foot in a grid path to find someone, but now you can use a drone thermal energy camera to pick that person out quickly.

Explore further: A better way to control a swarm of drones

Related Stories

A better way to control a swarm of drones

July 19, 2018

A team of researchers from Hungary, Norway and the Netherlands has found a way to better control a large swarm of drones—give them more autonomy. In their paper published in the journal Science Robotics, the group describes ...

Forget joysticks—use your torso to pilot drones

July 16, 2018

Imagine piloting a drone using the movements of your torso only and leaving your head free to look around, much like a bird. EPFL research has just shown that using your torso to pilot flying machines is indeed more immersive—and ...

Drones controlled with brain-computer interface

October 2, 2017

Single unmanned autonomous vehicles (UAVs) directed by joysticks, radio controllers, and mobile phones are already accomplishing a variety of useful tasks, such as aerial photography and security patrols. But using multiple ...

Robots learn by checking in on team members

June 13, 2018

The software and hardware needed to coordinate a team of unmanned aerial vehicles (UAVs) that can communicate and work toward a common goal have recently been developed by KAUST researchers.

Recommended for you

Coffee-based colloids for direct solar absorption

March 22, 2019

Solar energy is one of the most promising resources to help reduce fossil fuel consumption and mitigate greenhouse gas emissions to power a sustainable future. Devices presently in use to convert solar energy into thermal ...

NASA instruments image fireball over Bering Sea

March 22, 2019

On Dec. 18, 2018, a large "fireball—the term used for exceptionally bright meteors that are visible over a wide area—exploded about 16 miles (26 kilometers) above the Bering Sea. The explosion unleashed an estimated 173 ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.