Using smartphone cameras to track alertness

October 16, 2018 by Melanie Lefkowitz, Cornell University
Credit: CC0 Public Domain

Our level of alertness rises and falls over the course of a workday, sometimes causing our energy to drop and our minds to wander just as we need to perform important tasks.

To help understand these patterns and improve productivity, Cornell researchers have developed a tool that tracks by measuring size, captured through a burst of photographs taken every time users unlock their smartphones.

"Since our alertness fluctuates, if we can find a pattern it will be very useful to manage and schedule our day," said Vincent W.S. Tseng, a doctoral student in information science and lead author of "AlertnessScanner: What Do Your Pupils Tell About Your Alertness," presented in September at the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services.

Traditional methods of analyzing alertness tend to be cumbersome, often including devices that must be worn. Researchers in Cornell's People-Aware Computing Lab, run by Tanzeem Choudhury, associate professor of information science and senior author on the study, wanted to create a way to measure alertness unobtrusively and continuously.

"Since people use their phones very frequently during the day, we were thinking we could use phones as an instrument to understand and measure their alertness," Tseng said. "And since people's eyes are affected by their alertness, we were thinking that when people are looking at their phones, we could use a moment to measure their alertness at that point."

When people are alert, the sympathetic nervous system causes the pupils to dilate to make it easier to take in information. When they're drowsy, the parasympathetic nervous system causes the pupils to contract.

The paper, co-authored with Saeed Abdullah, an assistant professor in the College of Information Sciences and Technology at Pennsylvania State University, and Cornell doctoral student Jean Costa, included two studies conducted over two years. The first study analyzed results from 15 users, who were prompted to take photos of themselves every three hours. Their smartphones needed to have their infrared filters removed to make it easier to detect the contours of the pupil and the iris, particularly for people with dark eyes. The participants were also asked to complete a sleep journal, reporting how many hours they'd slept each night, and to take a phone-based Psychomotor Vigilance Test (PVT) – a five-minute quiz to gauge their reaction time – six times a day.

The photos gave researchers a view of participants' eyes that they then used to measure , making allowances for position and lighting, in order to predict a person's reaction time. This was then compared to the results from the PVT.

The researchers found that the pupil-scanning reliably predicted alertness. But because asking people to remove their phones' infrared filters was impractical, and prompting them to take photos of themselves throughout the day was too obtrusive, they conducted a second study a year later, when smartphone camera quality had improved enough that they no longer needed to remove the filters.

In that second study, eight participants were given smartphones with high-resolution front-facing cameras that took a burst of 30 photos in one second whenever the phones were unlocked. Users also completed the sleep journal and took the PVTs.

Though the two studies were difficult to compare because of their different methods, both showed that pupil scanning was a reliable means of predicting alertness. The second study, which took the photos passively in a burst, was deemed more practical because it required less work by the user, Tseng said.

Tseng said the AlertnessScanner could be particularly useful in health care, since medical professionals often work long hours doing intricate and important work. For example, clinicians typically look at devices during surgery, and a front-facing camera on the devices could track their alertness throughout procedures.

But understanding alertness patterns could be helpful to people in many kinds of workplaces, Tseng said.

"If you want to get something very important done, then probably you should execute this task while you're at the peak of your alertness; when you're in a valley of your alertness, you can do something like rote work," he said. "You'll also know the best time to take a break in order to allow your alertness or energy to go back up again."

Explore further: New algorithm determines ideal caffeine dosage and timing for alertness

More information: Vincent W.-S. Tseng et al. AlertnessScanner, Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI '18 (2018). DOI: 10.1145/3229434.3229456

Related Stories

Residents working long hours can increase alertness

September 12, 2018

(HealthDay)—Medical residents can take steps to maintain their energy and alertness during long shifts, according to an article published in the American Medical Association's AMA Wire.

Blue light may fight fatigue around the clock

February 3, 2014

Researchers from Brigham and Women's Hospital (BWH) have found that exposure to short wavelength, or blue light, during the biological day directly and immediately improves alertness and performance. These findings are published ...

Study shines light on night-time alertness

August 26, 2009

The circadian system is not the only pathway involved in determining alertness at night. Research described in the open access journal BMC Neuroscience showed that red light, which does not stimulate the circadian system, ...

Recommended for you

Uber filed paperwork for IPO: report

December 8, 2018

Ride-share company Uber quietly filed paperwork this week for its initial public offering, the Wall Street Journal reported late Friday.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.