This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication

trusted source


Researchers use AI to boost image quality of metalens camera

Researchers use artificial intelligence to boost image quality of metalens camera
Researchers used deep learning techniques to enhance the image quality of a camera with a metalens integrated directly onto a CMOS imaging chip (left). The metalens manipulates light using an array of 1000-nm tall cylindrical silicon nitride nano-posts (right). Credit: Ji Chen, Southeast University

Researchers have leveraged deep learning techniques to enhance the image quality of a metalens camera. The new approach uses artificial intelligence to turn low-quality images into high-quality ones, which could make these cameras viable for a multitude of imaging tasks including intricate microscopy applications and mobile devices.

Metalenses are ultrathin optical devices—often just a fraction of a millimeter thick—that use nanostructures to manipulate light. Although their could potentially enable extremely compact and lightweight cameras without traditional optical lenses, it has been difficult to achieve the necessary with these optical components.

"Our technology allows our metalens-based devices to overcome the limitations of image quality," said research team leader Ji Chen from Southeast University in China. "This advance will play an important role in the future development of highly portable consumer imaging electronics and can also be used in specialized imaging applications such as microscopy."

In the journal Optics Letters, the researchers describe how they used a type of known as a multi-scale convolutional neural network to improve resolution, contrast and distortion in images from a small —about 3 cm × 3 cm × 0.5 cm—they created by directly integrating a metalens onto a CMOS imaging chip.

"Metalens-integrated cameras can be directly incorporated into the imaging modules of smartphones, where they could replace the traditional refractive bulk lenses," said Chen. "They could also be used in devices such as drones, where the small size and lightweight camera would ensure imaging quality without compromising the drone's mobility."

Enhancing image quality

The camera used in the new work was previously developed by the researchers and uses a metalens with 1000-nm tall cylindrical silicon nitride nano-posts. The metalens focuses light directly onto a CMOS imaging sensor without requiring any other optical elements.

Although this design created a very small camera the compact architecture limited the image quality. Thus, the researchers decided to see if machine learning could be used to improve the images.

Deep learning is a type of machine learning that uses with multiple layers to automatically learn features from data and make complex decisions or predictions. The researchers applied this approach by using a convolution imaging model to generate a large number of high- and low-quality image pairs. These image pairs were used to train a multi-scale convolutional neural network so that it could recognize the characteristics of each type of image and use that to turn low-quality images into high-quality images.

"A key part of this work was developing a way to generate the large amount of training data needed for the neural network learning process," said Chen. "Once trained, a low-quality image can be sent from the device to into the neural network for processing, and high-quality imaging results are obtained immediately."

Researchers use artificial intelligence to boost image quality of metalens camera
The images show a comparison of ground truth, low-quality images, and neural network outputs for four test images. The first row represents simulation results, and the second row represents the experimental results. The blue, red and yellow boxes show close ups of details in the images. Credit: Ji Chen, Southeast University

Applying the neural network

To validate the new deep learning technique, the researchers used it on 100 test images. They analyzed two commonly used image processing metrics: the peak signal-to-noise ratio and the structural similarity index.

They found that the images processed by the neural network exhibited a significant improvement in both metrics. They also showed that the approach could rapidly generate high-quality imaging data that closely resembled what was captured directly through experimentation.

The researchers are now designing metalenses with complex functionalities—such as color or wide-angle imaging—and developing neural network methods for enhancing the imaging quality of these advanced metalenses. To make this technology practical for would require new assembly techniques for integrating metalenses into smartphone imaging modules and image quality enhancement software designed specifically for mobile phones.

"Ultra-lightweight and ultra-thin metalenses represent a revolutionary technology for future imaging and detection," said Chen. "Leveraging deep learning techniques to optimize metalens performance marks a pivotal developmental trajectory. We foresee machine learning as a vital trend in advancing photonics research."

More information: Yanxiang Zhang et al, Deep-learning enhanced high-quality imaging in metalens-integrated camera, Optics Letters (2024). DOI: 10.1364/OL.521393

Journal information: Optics Letters

Provided by Optica

Citation: Researchers use AI to boost image quality of metalens camera (2024, May 15) retrieved 25 May 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Researchers create miniature wide-angle camera with flat metalenses


Feedback to editors