Seeing the human pulse: Algorithm can accurately gauge heart rate by measuring tiny head movements in video data

Jun 20, 2013 by Larry Hardesty

Researchers at MIT's Computer Science and Artificial Intelligence Laboratory have developed a new algorithm that can accurately measure the heart rates of people depicted in ordinary digital video by analyzing imperceptibly small head movements that accompany the rush of blood caused by the heart's contractions.

In tests, the algorithm gave pulse measurements that were consistently within a few beats per minute of those produced by electrocardiograms (EKGs). It was also able to provide useful estimates of the time intervals between beats, a measurement that can be used to identify patients at risk for .

Guha Balakrishnan, a graduate student in MIT's Department of Electrical Engineering and Computer Science, and his two advisors—John Guttag, the Dugald C. Jackson Professor of Electrical Engineering and Computer Science and director of MIT's Data-Driven Medicine Group, and professor of and engineering Fredo Durand—describe the new algorithm in a paper appearing this summer at the Institute of Electrical and Electronics Engineers' and Pattern Recognition conference.

A video-based pulse- could be useful for monitoring newborns or the elderly, whose could be damaged by frequent attachment and removal of EKG leads. But, Guttag says, "From a medical perspective, I think that the long-term utility is going to be in applications beyond just pulse measurement."

For instance, Guttag says, an arterial obstruction could cause the blood to flow unevenly to the head. "Can you use the same type of techniques to look for bilateral ?" he asks. "What would it mean if you had more motion on one side than the other?"

Similarly, Guttag says, the technique could, in principle, measure cardiac output, or the volume of blood pumped by the heart, which is used in the diagnosis of several types of heart disease. Indeed, he says, before the advent of the echocardiogram, cardiac output was estimated by measuring exactly the types of mechanical forces that the new algorithm registers: In a technique called ballistocardiography, a heart patient would lie on a table with a low-friction suspension system; with every heartbeat, the table would move slightly, with a displacement corresponding to .

"I think this should be viewed as proof of concept," Guttag says. "It opens up a lot of potential flexibility."

Heart signals

The algorithm works by combining several techniques common in the field of computer vision. First, it uses standard face recognition to distinguish the subject's head from the rest of the image. Then it randomly selects 500 to 1,000 distinct points, clustered around the subjects' mouths and noses, whose movement it tracks from frame to frame. "I avoided the eyes, because there's blinking, which you don't want," Balakrishnan says.

Next, it filters out any frame-to-frame movements whose temporal frequency falls outside the range of a normal heartbeat—roughly 0.5 to 5 hertz, or 30 to 300 cycles per minute. That eliminates movements that repeat at a lower frequency, such as those caused by regular breathing and gradual changes in posture.

Finally, using a technique called principal component analysis, the algorithm decomposes the resulting signal into several constituent signals, which represent aspects of the remaining movements that don't appear to be correlated with each other. Of those signals, it selects the one that appears to be the most regular and that falls within the typical frequency band of the human pulse.

Balakrishnan also created a variation of the algorithm that doesn't use face recognition. Although its output was slightly less accurate, it was able to produce a reasonable approximation of pulse rate from video of the back of a subject's head.

The accuracy of the algorithm could also be improved by combining it with other video-analysis techniques, such as an algorithm that Guttag, Durand and several colleagues described last year, which amplifies otherwise imperceptible color changes between frames of video. "The signals can complement each other," Balakrishnan says.

Stephen Lewin-Berlin is managing director at Quanta Research Cambridge, a research subsidiary of the Taiwanese laptop manufacturer Quanta, which helped fund the research. Quanta's interest in visual measurement of vital signs grew out of a project to develop a commercial system for monitoring newborns being transferred by ambulance between hospitals; that system is now undergoing clinical trials in Taiwan.

"We had these videos of these patients and we were looking for ways of learning more about the infants based on the video signal," Lewin-Berlin says. "Their skin is very sensitive, so probes that attach to the skin that are not problematic for adult patients can actually cause damage to the patient."

"It's still, to some extent, an open question to understand exactly how some of the underlying biological functions—which include pumping blood and oxygenation and deoxygenation of hemoglobin and so on—are translated into measurable signals," Lewin-Berlin adds. "So I think there are still interesting opportunities in the medical domain to continue to explore what the different changes in the video signal mean and how they relate back to the underlying biological function."

Explore further: Computer scientists, electrical engineers devising algorithms that look for useful new patterns in medical sensor data

More information:

Related Stories

MIT group shows unseen motion captured in video

Mar 01, 2013

( —A baby lies in the crib looking motionless, a typical situation causing worry to new parents, wondering if the baby is still breathing. A video run through an algorithm designed for amplification ...

Guiding robot planes with hand gestures

Mar 15, 2012

Aircraft-carrier crew use a set of standard hand gestures to guide planes on the carrier deck. But as robot planes are increasingly used for routine air missions, researchers at MIT are working on a system that would enable ...

Recommended for you

Quantenna promises 10-gigabit Wi-Fi by next year

4 hours ago

( —Quantenna Communications has announced that it has plans for releasing a chipset that will be capable of delivering 10Gbps WiFi to/from routers, bridges and computers by sometime next year. ...

New US-Spanish firm says targets rich mobile ad market

4 hours ago

Spanish telecoms firm Telefonica and US investment giant Blackstone launched a mobile telephone advertising venture on Wednesday, challenging internet giants such as Google and Facebook in a multi-billion-dollar ...

Environmentally compatible organic solar cells

5 hours ago

Environmentally compatible production methods for organic solar cells from novel materials are in the focus of "MatHero". The new project coordinated by Karlsruhe Institute of Technology (KIT) aims at making ...

Twitter rules out Turkey office amid tax row

5 hours ago

Social networking company Twitter on Wednesday rejected demands from the Turkish government to open an office there, following accusations of tax evasion and a two-week ban on the service.

User comments : 1

Adjust slider to filter visible comments by rank

Display comments: newest first

1 / 5 (3) Jun 21, 2013
Some else results which never be published in MIT News. They are about stealing of both the ideas and money of taxpayers by numerous swindlers from David H. Koch Institute for Integrative Cancer Research and Department of Chemical Engineering, also with Department of Chemistry and Chemical Biology and School of Engineering and Applied Science of Harvard University at and
Their plagiaristic "masterpieces" titled Macroporous nanowire nanoelectronic scaffolds for synthetic tissues (DOI: 10.1038/NMAT3404) and Outside Looking In: Nanotube Transistor Intracellular Sensors ( were funded by NIH Director's Pioneer Award (1DP1OD003900) and a McKnight Foundation Technological Innovations in Neurosciences Award, also a Biotechnology Research Endowment from the Dep. of Anesthesiology at Children's Hospital Boston and NIH grant GM073626, and NIH grants DE013023 and DE016516.

More news stories

Quantenna promises 10-gigabit Wi-Fi by next year

( —Quantenna Communications has announced that it has plans for releasing a chipset that will be capable of delivering 10Gbps WiFi to/from routers, bridges and computers by sometime next year. ...

Floating nuclear plants could ride out tsunamis

When an earthquake and tsunami struck the Fukushima Daiichi nuclear plant complex in 2011, neither the quake nor the inundation caused the ensuing contamination. Rather, it was the aftereffects—specifically, ...

Unlocking secrets of new solar material

( —A new solar material that has the same crystal structure as a mineral first found in the Ural Mountains in 1839 is shooting up the efficiency charts faster than almost anything researchers have ...

Patent talk: Google sharpens contact lens vision

( —A report from Patent Bolt brings us one step closer to what Google may have in mind in developing smart contact lenses. According to the discussion Google is interested in the concept of contact ...

How kids' brain structures grow as memory develops

Our ability to store memories improves during childhood, associated with structural changes in the hippocampus and its connections with prefrontal and parietal cortices. New research from UC Davis is exploring ...