Communicating with drones using facial expressions and gestures could help save lives

With first responders beginning to use robots and drones to assist in search and rescue operations, having simple and easy to use ways of communicating with them can save precious moments and could help save lives.

In Simon Fraser University's Autonomy Lab, researchers are using artificial intelligence to develop drone technologies that can make the human-robot interaction more intuitive, without requiring the standard controller.

"Most commercial drones today come with controllers which work really well, but sometimes you may find yourself in a situation where your hands are busy," notes computing science professor Richard Vaughan, who leads this research.

"Or maybe you weren't expecting to interact with a drone today so you don't have special equipment with you. We'd like to be able to command drones in these situations and make the interaction natural and intuitive."

Vaughan's team is building a drone whose flight direction can be controlled by arm gestures—you can even command the drone to do a flip in mid-air.

Another drone the team has developed is one that executes commands based on the user's facial expressions. By recognizing a "trigger-face" the drone could be used to carry out actions such as taking a photo or video of an inaccessible area to assess a situation.

"We would like to get to the point where interacting with a robot is as easy as working with a co-worker or a trained animal," says Vaughan.

As robot technology continues to advance, it is likely that one day we will be interacting with robots in our everyday life. Being able to communicate and provide commands to them without a controller is not only convenient but can help navigate emergency situations.

Fast Facts:

• Fire Fighters in the U.S. are already using drones for emergency scenarios like fires and search and rescue missions.
• Drones can be deployed very quickly. What used to take 10 or 15 minutes with just humans will only take 2-3 minutes with drones.
• Currently, search and rescue teams send out 100 people out to walk on a foot in a grid path to find someone, but now you can use a drone thermal energy camera to pick that person out quickly.