Communicating with drones using facial expressions and gestures could help save lives

July 25, 2018, Simon Fraser University
Computing science professor Richard Vaughan (right) and Ph.D. student Sepehr MohaimenianPour (left) are developing technologies in SFU's Autonomy Lab to help users interact with drones in a more intuitive manner. Credit: Simon Fraser University

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.

In Simon Fraser University's Autonomy Lab, researchers are using artificial intelligence to develop technologies that can make the more intuitive, without requiring the standard controller.

"Most commercial drones today come with controllers which work really well, but sometimes you may find yourself in a situation where your hands are busy," notes computing science professor Richard Vaughan, who leads this research.

"Or maybe you weren't expecting to interact with a drone today so you don't have special equipment with you. We'd like to be able to command drones in these situations and make the interaction natural and intuitive."

Vaughan's team is building a drone whose flight direction can be controlled by arm gestures—you can even command the drone to do a flip in mid-air.

Another drone the team has developed is one that executes commands based on the user's facial expressions. By recognizing a "trigger-face" the drone could be used to carry out actions such as taking a photo or video of an inaccessible area to assess a situation.

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.In Simon Fraser University's Autonomy Lab, researchers are using artificial intelligence to develop drone technologies that can make the human-robot interaction more intuitive, without requiring the standard controller. Credit: Simon Fraser University
"We would like to get to the point where interacting with a is as easy as working with a co-worker or a trained animal," says Vaughan.

As robot technology continues to advance, it is likely that one day we will be interacting with robots in our everyday life. Being able to communicate and provide commands to them without a controller is not only convenient but can help navigate emergency situations.

Fast Facts:

  • Fire Fighters in the U.S. are already using drones for emergency scenarios like fires and search and rescue missions.
  • Drones can be deployed very quickly. What used to take 10 or 15 minutes with just humans will only take 2-3 minutes with drones.
  • Currently, search and rescue teams send out 100 people out to walk on a foot in a grid path to find someone, but now you can use a drone thermal energy camera to pick that person out quickly.

Explore further: A better way to control a swarm of drones

Related Stories

A better way to control a swarm of drones

July 19, 2018

A team of researchers from Hungary, Norway and the Netherlands has found a way to better control a large swarm of drones—give them more autonomy. In their paper published in the journal Science Robotics, the group describes ...

Forget joysticks—use your torso to pilot drones

July 16, 2018

Imagine piloting a drone using the movements of your torso only and leaving your head free to look around, much like a bird. EPFL research has just shown that using your torso to pilot flying machines is indeed more immersive—and ...

Drones controlled with brain-computer interface

October 2, 2017

Single unmanned autonomous vehicles (UAVs) directed by joysticks, radio controllers, and mobile phones are already accomplishing a variety of useful tasks, such as aerial photography and security patrols. But using multiple ...

Robots learn by checking in on team members

June 13, 2018

The software and hardware needed to coordinate a team of unmanned aerial vehicles (UAVs) that can communicate and work toward a common goal have recently been developed by KAUST researchers.

Recommended for you

Privacy becomes a selling point at tech show

January 7, 2019

Apple is not among the exhibitors at the 2019 Consumer Electronics Show, but that didn't prevent the iPhone maker from sending a message to attendees on a large billboard.

China's Huawei unveils chip for global big data market

January 7, 2019

Huawei Technologies Ltd. showed off a new processor chip for data centers and cloud computing Monday, expanding into new and growing markets despite Western warnings the company might be a security risk.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.