Engineers design video game controller that can sense players' emotions (w/ video)

April 8, 2014 by Bjorn Carey, Stanford University

( —Stanford engineers have developed what could be the next big thing in interactive gaming: handheld game controllers that measure the player's physiology and alter the gameplay to make it more engaging.

Sometimes, a dozen ravenous zombies just aren't exciting enough to hold a video gamer's interest. The next step in , however, could come in the form of a handheld game controller that gauges the player's brain activity and throws more zombies on the screen when it senses the player is bored.

The prototype controller was born from research conducted in the lab of Gregory Kovacs, a professor of electrical engineering at Stanford, in collaboration with Texas Instruments. The main area of research by grad students in Kovacs' lab involves developing practical ways of measuring physiological signals to determine how a person's bodily systems are functioning.

One such system of interest to Corey McCall, a doctoral candidate in Kovacs' lab, is the autonomic nervous system, the emotional part of the brain – the part that changes when you get excited or bored, happy or sad. This activity, in turn, influences your heart rate, respiration rate, temperature, perspiration and other key bodily processes. Measuring these outward signs offers a way of reverse engineering what's occurring in the brain.

"You can see the expression of a person's in their heart rate and and respiration rate, and by measuring those outputs, we can understand what's happening in the brain almost instantaneously," said McCall, the leader on the project.

This method of sensing autonomic activity is particularly intriguing, McCall said, because it can be conducted via non-invasive means. In fact, another of his projects involves monitoring the skin temperature of epilepsy patients at Stanford Hospital in an effort to sense the early indicators of a seizure.

As McCall worked out other ways to measure autonomic activity, he realized that he could easily monitor people in various mental states as they played video games and that he could gather most of the data he needed straight from test subjects' hands.

McCall popped the back panel off an Xbox 360 controller and replaced it with a 3-D printed plastic module packed with sensors. Small metal pads on the controller's surface measure the user's heart rate, blood flow, and both the rate of breath and how deeply the user is breathing. Another light-operated sensor gives a second measurement, and accelerometers measure how frantically the person is shaking the controller.

Meanwhile, custom-built software gauges the intensity of the game – a simple but fast-paced racing game in which the player must drive over colored tiles in a particular sequence. McCall can then compare all this data to generate an overall picture of the player's level of mental engagement.

The controller received a lot of positive interest when McCall presented it at the International Consumer Electronics Show in Las Vegas in January, in part because of the next phase of the work: Use the controller to provide feedback to the gaming console, which can then alter the pace of gameplay to better suit the player.

"If a player wants maximum engagement and excitement, we can measure when they are getting bored and, for example, introduce more zombies into the level," McCall said. "We can also control the game for children. If parents are concerned that their children are getting too wrapped up in the game, we can tone it down or remind them that it's time for a healthy break."

Explore further: Pico projector used in eye based video gaming system

Related Stories

Pico projector used in eye based video gaming system

May 3, 2011

( -- Students at the University of Texas in Austen are playing video games. Honestly, that is really not news. Students all over the country are playing video games, usually when they should be studying. In this ...

Nintendo gives 2nd glimpse of Wii U game machine

January 10, 2012

Nintendo Co.'s upcoming Wii U game console will come with a controller that has a big, touch-enabled screen. At first glance, that seems like an obstacle to the kind of casual multiplayer gaming that made the first Wii console ...

Review: Shark 360 mouse offers added precision

January 4, 2012

Console gaming occasionally gets a boost from add-ons that can make the standard controller more efficient to hold and operate. A new wireless mouse looks to replace the standard Xbox 360 controller altogether in hopes of ...

Review: Xbox One vs. PlayStation 4

November 29, 2013

If you're a serious gamer, these are exciting days. If you're not a serious gamer, you can be excused for wondering what the fuss is all about.

Recommended for you

Privacy becomes a selling point at tech show

January 7, 2019

Apple is not among the exhibitors at the 2019 Consumer Electronics Show, but that didn't prevent the iPhone maker from sending a message to attendees on a large billboard.

China's Huawei unveils chip for global big data market

January 7, 2019

Huawei Technologies Ltd. showed off a new processor chip for data centers and cloud computing Monday, expanding into new and growing markets despite Western warnings the company might be a security risk.


Adjust slider to filter visible comments by rank

Display comments: newest first

Theodor Roll
not rated yet Apr 08, 2014
Reminds me of the Nintendo 64 Bio Sensor (from 1998), which measured Heart Rate aswell. You could use it with Tetris 64. This one seems to have a few more features though.
not rated yet Apr 08, 2014
This concept has a future in game difficulty scaling systems for sure.
not rated yet Apr 09, 2014
The only thing I need a controller to sense is my input and they suck at that let alone other BS. For instance on a PS3 controller both sticks have a button when pressed down (L3 and R3). The sticks have flat tops requiring you to press down just to move them. Incredibly stupid design, you shouldn't have to press down to move a stick when pressing down activates a different function. It's like having a squeeze kill switch on your steering wheel and you have to squeeze the wheel to turn it. Can't tell you how many times that button has killed me in the heat of combat. Since the idiots at sony don't mandate reconfigurable controls in games you're usually stuck with whatever function is on them.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.