Team creates click-and-drag interface enabling rapid video object segmentation

June 12, 2015, Disney Research

Investigators at Disney Research Zurich have developed a method for achieving very accurate object segmentation of video by enabling human editors to work efficiently with state-of-the-art algorithms using a click-and-drag interface.

Segmentation, which identifies objects, backgrounds and other meaningful regions within an image or , is a necessary step for many editing tasks and for image analysis. People can readily perceive objects and the composition of scenes despite variations in colors, lighting and contours, but despite significant advances in recent years, automatic image is still inexact.

By keeping a human in the loop, a semi-automatic segmentation system is able to achieve much higher accuracies than possible with fully automated systems, said Miquel A. Farré, associate research engineer.

The interface he and Jordi Pont-Tuset and Aljoscha Smolic developed allows human editors to work with computer software to rapidly identify the regions to be segmented in a video image and then propagate those segmentations in subsequent video frames. They will present their method at the International Workshop on Content-Based Multimedia Indexing, June 10-12 in Prague, Czech Republic.

To minimize the amount of necessary to achieve an accurate result, the researchers used an approach called segmentation hierarchy to represent images. It transforms an image from a matrix of pixels to a tree-like graph whose nodes represent regions of the images, from small detailed regions in the graph's "leaves" to the whole image in the root.

The software automatically builds these tree-like graphs and the human editor, using the click-and-drag interface, selects and annotates the combination of nodes that represent the object or other image element of interest. The software tracks these nodes - as well as the nodes that the editor avoided - in subsequent frames of the video, with the editor making corrections and adjustments as necessary. This method optimized the quality of the segmentation, they said, while minimizing the amount of human interaction.

The researchers evaluated their method by comparing its performance with 88 sequences obtained from a standard video dataset that had been segmented by human editors. They found that the quality achieved by the semi-automatic method was similar in most cases to that of manual segmentation while requiring much less time.

Explore further: Simulating subsurface flow and transport at multiple scales

More information: Semi-Automatic Video Object Segmentation by Advanced Manipulation of Segmentation Hierarchies-Paper, www.disneyresearch.com/wp-cont … ierarchies-Paper.pdf

Related Stories

Tumor segmentation software receives 510k clearance from FDA

October 9, 2014

Columbia University's imaging software that facilitates 3-D lung tumor segmentation, licensed to Varian Medical Systems, has been incorporated into the Smart Segmentation module of Varian's Eclipse treatment planning system ...

Team develops advanced video and image processing

March 27, 2013

Rapid developments in satellite and sensor technologies have increased the availability of high-resolution, remotely sensed images faster than researchers can process and analyze the data manually.

Recommended for you

What can snakes teach us about engineering friction?

May 21, 2018

If you want to know how to make a sneaker with better traction, just ask a snake. That's the theory driving the research of Hisham Abdel-Aal, Ph.D., an associate teaching professor from Drexel University's College of Engineering ...

Flexible, highly efficient multimodal energy harvesting

May 21, 2018

A 10-fold increase in the ability to harvest mechanical and thermal energy over standard piezoelectric composites may be possible using a piezoelectric ceramic foam supported by a flexible polymer support, according to Penn ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.