Seeing the human pulse: Algorithm can accurately gauge heart rate by measuring tiny head movements in video data

June 20, 2013 by Larry Hardesty

Researchers at MIT's Computer Science and Artificial Intelligence Laboratory have developed a new algorithm that can accurately measure the heart rates of people depicted in ordinary digital video by analyzing imperceptibly small head movements that accompany the rush of blood caused by the heart's contractions.

In tests, the algorithm gave pulse measurements that were consistently within a few beats per minute of those produced by electrocardiograms (EKGs). It was also able to provide useful estimates of the time intervals between beats, a measurement that can be used to identify patients at risk for .

Guha Balakrishnan, a graduate student in MIT's Department of Electrical Engineering and Computer Science, and his two advisors—John Guttag, the Dugald C. Jackson Professor of Electrical Engineering and Computer Science and director of MIT's Data-Driven Medicine Group, and professor of and engineering Fredo Durand—describe the new algorithm in a paper appearing this summer at the Institute of Electrical and Electronics Engineers' and Pattern Recognition conference.

A video-based pulse- could be useful for monitoring newborns or the elderly, whose could be damaged by frequent attachment and removal of EKG leads. But, Guttag says, "From a medical perspective, I think that the long-term utility is going to be in applications beyond just pulse measurement."

For instance, Guttag says, an arterial obstruction could cause the blood to flow unevenly to the head. "Can you use the same type of techniques to look for bilateral ?" he asks. "What would it mean if you had more motion on one side than the other?"

Similarly, Guttag says, the technique could, in principle, measure cardiac output, or the volume of blood pumped by the heart, which is used in the diagnosis of several types of heart disease. Indeed, he says, before the advent of the echocardiogram, cardiac output was estimated by measuring exactly the types of mechanical forces that the new algorithm registers: In a technique called ballistocardiography, a heart patient would lie on a table with a low-friction suspension system; with every heartbeat, the table would move slightly, with a displacement corresponding to .

"I think this should be viewed as proof of concept," Guttag says. "It opens up a lot of potential flexibility."

Heart signals

The algorithm works by combining several techniques common in the field of computer vision. First, it uses standard face recognition to distinguish the subject's head from the rest of the image. Then it randomly selects 500 to 1,000 distinct points, clustered around the subjects' mouths and noses, whose movement it tracks from frame to frame. "I avoided the eyes, because there's blinking, which you don't want," Balakrishnan says.

Next, it filters out any frame-to-frame movements whose temporal frequency falls outside the range of a normal heartbeat—roughly 0.5 to 5 hertz, or 30 to 300 cycles per minute. That eliminates movements that repeat at a lower frequency, such as those caused by regular breathing and gradual changes in posture.

Finally, using a technique called principal component analysis, the algorithm decomposes the resulting signal into several constituent signals, which represent aspects of the remaining movements that don't appear to be correlated with each other. Of those signals, it selects the one that appears to be the most regular and that falls within the typical frequency band of the human pulse.

Balakrishnan also created a variation of the algorithm that doesn't use face recognition. Although its output was slightly less accurate, it was able to produce a reasonable approximation of pulse rate from video of the back of a subject's head.

The accuracy of the algorithm could also be improved by combining it with other video-analysis techniques, such as an algorithm that Guttag, Durand and several colleagues described last year, which amplifies otherwise imperceptible color changes between frames of video. "The signals can complement each other," Balakrishnan says.

Stephen Lewin-Berlin is managing director at Quanta Research Cambridge, a research subsidiary of the Taiwanese laptop manufacturer Quanta, which helped fund the research. Quanta's interest in visual measurement of vital signs grew out of a project to develop a commercial system for monitoring newborns being transferred by ambulance between hospitals; that system is now undergoing clinical trials in Taiwan.

"We had these videos of these patients and we were looking for ways of learning more about the infants based on the video signal," Lewin-Berlin says. "Their skin is very sensitive, so probes that attach to the skin that are not problematic for adult patients can actually cause damage to the patient."

"It's still, to some extent, an open question to understand exactly how some of the underlying biological functions—which include pumping blood and oxygenation and deoxygenation of hemoglobin and so on—are translated into measurable signals," Lewin-Berlin adds. "So I think there are still interesting opportunities in the medical domain to continue to explore what the different changes in the video signal mean and how they relate back to the underlying biological function."

Explore further: MIT group shows unseen motion captured in video

More information: … efromheadmotion.html

Related Stories

MIT group shows unseen motion captured in video

March 1, 2013

( —A baby lies in the crib looking motionless, a typical situation causing worry to new parents, wondering if the baby is still breathing. A video run through an algorithm designed for amplification shows the baby ...

Guiding robot planes with hand gestures

March 15, 2012

Aircraft-carrier crew use a set of standard hand gestures to guide planes on the carrier deck. But as robot planes are increasingly used for routine air missions, researchers at MIT are working on a system that would enable ...

Recommended for you

Volvo to supply Uber with self-driving cars (Update)

November 20, 2017

Swedish carmaker Volvo Cars said Monday it has signed an agreement to supply "tens of thousands" of self-driving cars to Uber, as the ride-sharing company battles a number of different controversies.

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (3) Jun 21, 2013
Some else results which never be published in MIT News. They are about stealing of both the ideas and money of taxpayers by numerous swindlers from David H. Koch Institute for Integrative Cancer Research and Department of Chemical Engineering, also with Department of Chemistry and Chemical Biology and School of Engineering and Applied Science of Harvard University at and
Their plagiaristic "masterpieces" titled Macroporous nanowire nanoelectronic scaffolds for synthetic tissues (DOI: 10.1038/NMAT3404) and Outside Looking In: Nanotube Transistor Intracellular Sensors ( were funded by NIH Director's Pioneer Award (1DP1OD003900) and a McKnight Foundation Technological Innovations in Neurosciences Award, also a Biotechnology Research Endowment from the Dep. of Anesthesiology at Children's Hospital Boston and NIH grant GM073626, and NIH grants DE013023 and DE016516.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.