Researchers use AI to dramatically increase image clarity under severe conditions

June 5, 2017
NEC Corporation and Tokyo Institute of Technology (Tokyo Tech) today announced their joint development of a multi-modal image fusion technology that dramatically improves the clarity of images by using artificial intelligence (AI) to automatically combine visible images taken by standard cameras with non-visible images taken by specialized devices, such as thermal or terahertz cameras. Credit: Obtained permission to use the severe environment simulator in the ImPACT Tough Robotics Challenge program.

In recent years, technological advancements and cost reductions for specialized devices that photograph non-visible images, such as thermal cameras, which capture heat, and X-ray cameras, have enabled them to be used for an expanding range of purposes, including nighttime monitoring under severe weather conditions, such as dense fog. However, these cameras tend to provide images of poor resolution and quality in comparison to images taken of visible subjects.

Conventionally, in order to analyze non-visible and visible images of the same subject, images had to be viewed and compared separately, or they had to be manually combined by a trained expert. Moreover, the detection of abnormalities or hazards contained in non-visible images could be easily overlooked when combining the images.

This new technology uses AI to achieve greater image visibility by automatically selecting highly visible parts from multiple images and combining them, while enhancing the smallest characteristics contained in non-visible images. Specifically, AI carries out detailed examination of each image in order to assess the degree of visibility of each part, then automatically extracts the best areas from each image, taking environmental characteristics into consideration, such as brightness, the direction of light and obstacles.

"As this technology enables instant visual clarification, even under harsh conditions, it allows users to make well informed evaluations. For example, it can be applied to monitoring systems to assist with nighttime observations, or to infrastructure inspection devices to improve the detection of interior and exterior abnormalities, such as cracking," said Akio Yamada, General Manager, Data Science Research Laboratories, NEC Corporation.

Credit: Obtained permission to use the severe environment simulator in the ImPACT Tough Robotics Challenge program.

"Until now, a specialist had to manually carry out complex conversion tasks in order to combine images taken by different types of cameras. This technology eliminates the need for such manual work, using AI to effectively and automatically combine images taken by different cameras. This also increases visibility by actively utilizing the strong points of each and non-visible image, even when the images are difficult to visualize," said Professor Masatoshi Okutomi, School of Engineering, Tokyo Institute of Technology.

Moreover, AI analyzes the slightest clue to detecting an abnormality and hazard in a non-visible image and automatically generates a multimodal (fusion of visible and non-visible) image with exceptionally high visibility, while properly regulating the degree of enhancement to avoid causing image breakdowns, such as clipped highlights and crushed blacks.

NEC and Tokyo Tech will present this on June 7, at the 23rd Symposium on Sensing via Image Information to be held at Pacifico Yokohama (Nishi-ku, Yokohama City) from Wednesday, June 7 to Friday, June 9.

Credit: Obtained permission to use the severe environment simulator in the ImPACT Tough Robotics Challenge program.

Explore further: Image: Sequence of Juno spacecraft's close approach to Jupiter

Related Stories

How a Hubble image goes from photons to finished beauty

June 10, 2013

How does raw data from the Hubble Space Telescope end up to become a finished gorgeous color image, like the one of Arp 274, above? It's an interesting process, because the cameras on Hubble do not take color pictures.

Fast and accurate infrared 3-D scanner

May 2, 2017

Infrared 3-D scanners have been used in video games for quite some time. Whereas in video games the scanners are, for example, only able to identify if a player throws his arms up in the air while playing virtual volleyball, ...

Recommended for you

Enhancing solar power with diatoms

October 20, 2017

Diatoms, a kind of algae that reproduces prodigiously, have been called "the jewels of the sea" for their ability to manipulate light. Now, researchers hope to harness that property to boost solar technology.

Dutch open 'world's first 3D-printed bridge'

October 17, 2017

Dutch officials toasted on Tuesday the opening of what is being called the world's first 3D-printed concrete bridge, which is primarily meant to be used by cyclists.

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.