Computational imaging on the electric grid

July 14, 2017, American Technion Society
Hallway scene: click to see digital separation to components and relighting. Credit: American Technion Society

Researchers at the Technion-Israel Institute of Technology and the University of Toronto have developed a technology for producing a new understanding of the nighttime landscape—from the office level to the entire city—based on the flicker of electric lights.

Artificial lighting plays a central role in our lives—in the home, in the office, on the road, and more—and is produced by a variety of lamps that are found in streetlights, offices, searchlights, billboards, computer monitors and more. Light emitted from all lamps connected to the electricity grid is constantly changing, but because of the high speed of this effect, people do not sense this flickering.

In a study to be presented July 22nd at the Computer Vision and Pattern Recognition conference of the Institute of Electrical and Electronics Engineers (IEEE), lead researcher Mark Sheinin of the Technion's Andrew and Erna Viterbi Faculty of Electrical Engineering, along with Technion Professor Yoav Schechner and Professor Kyros Kutulakos of the University of Toronto will present a new way to produce a great deal of useful information from the flicker patterns of lighted scenes. The approach combines various fields of research, including optics, computer vision, image processing and engineering.

The researchers developed a system that extracts information from a passive video (without additional lighting) of the desired scene—office, hallway, even an entire city. The analysis of the information obtained from the photograph concludes, among other things, how the scene would look if some of the bulbs were turned off, amplified or replaced them with a different type of , which may also help to cancel the reflections from windows. The analysis could help people take "selfies" in a restaurant and digitally eliminate a shadow-casting ceiling bulb, and vary the color and shading in the background.

Furthermore, the researchers found that the flicker across city scale provides valuable information about the electric grid itself, with potential to indicate anomalies in its dynamics. Based on this industrial application, they thus submitted a patent jointly with Technion Professor Yoash Levron.

The reason for the flicker of the light is that electrical networks operate in an alternating current (AC), in which the current of the electrons continuously reverses its direction. In North America, for example, the frequency in the grid is 60 Hz, which means that the electron current changes direction 120 times per second, and that is the rate at which the light flickers. The jitter pattern depends on the type of bulb—fluorescent, mercury, halogen, LED—since each bulb converts the electricity energy to light in a different process; in other words, each type of bulb has a unique time signature.

For photographers, flicker photography is a challenge. On the one hand, identifying the dynamics of flicker requires a very brief exposure. On the other hand, photography at night requires a long exposure to collect enough light to create an image. To solve this discrepancy, the researchers developed a unique electro-optical camera called ACam to sense the flicker of the alternating current. The camera, which is connected to the , uses the flicker cycle to capture fast signals from the scene. The camera's electronic shutter is open the entire time an image is taken, but the scene is only visible to the sensor in the desired time section of each flicker cycle.

The technology has developed a pathway for further research to perform a wide range of tasks, including the controlled illumination of objects, the measurement of three-dimensional objects and their surface texture based on their shadow, and the analysis of the properties of the electrical grid remotely by optical methods.

Srinivas Narasimhan, a professor in the School of Computer Sciences at Carnegie-Mellon University who was not involved with the study, said the ACam paper demonstrates "really innovative work, by measuring a signal that is hidden in plain sight and turning it into useful information. Its applications could include light pollution monitoring, air quality estimation at night, non-line-of-sight imaging, and monitoring power grid output and fluctuations."

Professor Schechner's interest in amateur astronomy, and the challenge light pollution poses for city star-gazers, prompted an interest in studying these subtle changes in light. Street lights flicker, so he reasoned "a way to capture the flicker could allow us to make urban night-sky observations, by integrating the brief moments that the light flicker has minimum brightness."

But the researchers soon discovered that light bulbs don't all in the same way. "When one bulb reaches a minimum brightness, another bulb might be approaching its own maximum."

"So the astronomical idea was shelved for the time being," Professor Schechner added, "but we found great new uncharted territory to explore: the electrical grid."

Explore further: "Virtual periscope" sees above-surface, airborne objects from underwater view

More information: Mark Sheinin, Yoav Y. Schechner and Kiriakos. N. Kutulakos, "Computational imaging on the electric grid," To appear in Proc. IEEE CVPR (2017) … ations/ACam_CVPR.pdf

Mark Sheinin, Yoav Y. Schechner and Kiriakos. N. Kutulakos, "Computational imaging on the electric grid: Supplementary material," Supplemental document in Proc. IEEE CVPR (2017), describing the DELIGHT database and some technical aspects. … ent_ACam_docCVPR.pdf

Related Stories

Recommended for you

Tiny 'water bears' can teach us about survival

March 20, 2019

Earth's ultimate survivors can weather extreme heat, cold, radiation and even the vacuum of space. Now the U.S. military hopes these tiny critters called tardigrades can teach us about true toughness.

A decade on, smartphone-like software finally heads to space

March 20, 2019

Once a traditional satellite is launched into space, its physical hardware and computer software stay mostly immutable for the rest of its existence as it orbits the Earth, even as the technology it serves on the ground continues ...

Researchers find hidden proteins in bacteria

March 20, 2019

Scientists at the University of Illinois at Chicago have developed a way to identify the beginning of every gene—known as a translation start site or a start codon—in bacterial cell DNA with a single experiment and, through ...

Turn off a light, save a life, says new study

March 20, 2019

We all know that turning off lights and buying energy-efficient appliances affects our financial bottom line. Now, according to a new study by University of Wisconsin-Madison researchers, we know that saving energy also saves ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.