Seamless photography: Using mathematical models for image stitching

Oct 02, 2013
This shows two input images (from left) stitched together using the proposed method (right). Credit: original image: Blanka M. Lukes, Prague Private Guides; modified image: Wei Wang and Michael Ng.

A photo captures only as much as the camera in use will allow, and is therefore limited by the field of view of the camera's lens. In the case of smartphones and many advanced cameras, the view from the lens is much smaller than the view from your own eyes.

Panoramic photographs were invented to capture large objects or scenes that could not otherwise fit within the constraints of a single photo. Panoramic photography is achieved through image stitching, a process that combines two or more photographs, seamlessly blending input with overlapping regions into one picture. A paper published by Wei Wang and Michael Ng in the SIAM Journal on Imaging Sciences this summer aims to develop an algorithm for image stitching.

Image stitching involves two steps: image alignment and image blending. Image alignment finds point pairs in the overlapping region of two images that correspond to one another. Image blending combines the two aligned images seamlessly. This step is important if the pixel intensities in the different images vary enough to produce artifacts such as varying lighting conditions and different exposure settings. In this paper, the authors focus on image blending, assuming that the images have been aligned.

Many different approaches for image blending are seen in the literature. "The traditional method is to search for a curve in the overlapping area in which the differences among the input images are minimal," explains author Michael Ng. "However, the curve may not be determined accurately because of light intensity, color inconsistency, parallax, occlusion, etc."

The approach used in this paper instead minimizes seam artifacts by smoothing the transition between the images. The mosaic image here is a weighted combination of the input images. This means the pixel values from the two overlapping images are combined using a weighted average for qualities such as exposure, local contrast, saturation, etc.

How is this achieved?

Many systems, both natural and man-made, seek out the lowest energy state, such as, a ball rolling down a hill, or a snow-laden tree branch bending to maintain the lowest possible energy in the system. The concept of minimizing the energy of a given system is also used in image processing. For a given image, an energy function is defined and minimized to get a better image (i.e. less noise, better sharpness, higher contrast, etc.). This is the approach the authors use in the paper. Seamless combination of images is achieved by minimizing an energy function based on intensity or gradient differences of the two images.

"According to the model, we construct a weighting function over the overlapping area so that a panoramic image can be generated," says Ng. "The optimal weighting function can be obtained by minimizing the overall energy of the mathematical model." Thus, in the proposed model, both the weighting function and the ?nal blending in the overlapping region are based on solving an energy minimizing problem. The authors show how to define an energy function and develop an algorithm to minimize it.

This variational method—based on achieving the lowest energy or ground state—is seen to produce a more visually appealing photo in comparison to other existing methods.

Future work may extend the scope of this research beyond two-dimensional images. "It is interesting to consider extending the current variational approach to tackle the problem of three-dimensional image stitching in medical imaging applications and stitching video in computer applications," Ng says.

Explore further: Special CT improves radiotherapy planning

More information: A Variational Approach for Image Stitching I, Wei Wang and Michael K. Ng SIAM Journal on Imaging Sciences, 6(3), 1318 (Online publish date: 11 July 2013). epubs.siam.org/doi/abs/10.1137/110819871

To read related research published by the authors about further developing an image stitching algorithm using the gradients of input images, view: A Variational Approach for Image Stitching II: Using Image Gradients epubs.siam.org/doi/abs/10.1137/120872140 SIAM Journal on Imaging Sciences, 6(3), 1345

Related Stories

Special CT improves radiotherapy planning

Sep 19, 2013

A computer tomograph (CT) with special software solutions is helping to improve the planning of radiotherapy, thus making cancer treatment more effective. Radiotherapy aims to irradiate tumors as precisely ...

Seeing depth through a single lens

Aug 05, 2013

Researchers at the Harvard School of Engineering and Applied Sciences (SEAS) have developed a way for photographers and microscopists to create a 3D image through a single lens, without moving the camera.

Recommended for you

Enabling a new future for cloud computing

Aug 21, 2014

The National Science Foundation (NSF) today announced two $10 million projects to create cloud computing testbeds—to be called "Chameleon" and "CloudLab"—that will enable the academic research community ...

Hacking Gmail with 92 percent success

Aug 21, 2014

(Phys.org) —A team of researchers, including an assistant professor at the University of California, Riverside Bourns College of Engineering, have identified a weakness believed to exist in Android, Windows ...

User comments : 0