Team creates click-and-drag interface enabling rapid video object segmentation

June 12, 2015, Disney Research

Investigators at Disney Research Zurich have developed a method for achieving very accurate object segmentation of video by enabling human editors to work efficiently with state-of-the-art algorithms using a click-and-drag interface.

Segmentation, which identifies objects, backgrounds and other meaningful regions within an image or , is a necessary step for many editing tasks and for image analysis. People can readily perceive objects and the composition of scenes despite variations in colors, lighting and contours, but despite significant advances in recent years, automatic image is still inexact.

By keeping a human in the loop, a semi-automatic segmentation system is able to achieve much higher accuracies than possible with fully automated systems, said Miquel A. Farré, associate research engineer.

The interface he and Jordi Pont-Tuset and Aljoscha Smolic developed allows human editors to work with computer software to rapidly identify the regions to be segmented in a video image and then propagate those segmentations in subsequent video frames. They will present their method at the International Workshop on Content-Based Multimedia Indexing, June 10-12 in Prague, Czech Republic.

To minimize the amount of necessary to achieve an accurate result, the researchers used an approach called segmentation hierarchy to represent images. It transforms an image from a matrix of pixels to a tree-like graph whose nodes represent regions of the images, from small detailed regions in the graph's "leaves" to the whole image in the root.

The software automatically builds these tree-like graphs and the human editor, using the click-and-drag interface, selects and annotates the combination of nodes that represent the object or other image element of interest. The software tracks these nodes - as well as the nodes that the editor avoided - in subsequent frames of the video, with the editor making corrections and adjustments as necessary. This method optimized the quality of the segmentation, they said, while minimizing the amount of human interaction.

The researchers evaluated their method by comparing its performance with 88 sequences obtained from a standard video dataset that had been segmented by human editors. They found that the quality achieved by the semi-automatic method was similar in most cases to that of manual segmentation while requiring much less time.

Explore further: Simulating subsurface flow and transport at multiple scales

More information: Semi-Automatic Video Object Segmentation by Advanced Manipulation of Segmentation Hierarchies-Paper, www.disneyresearch.com/wp-cont … ierarchies-Paper.pdf

Related Stories

Tumor segmentation software receives 510k clearance from FDA

October 9, 2014

Columbia University's imaging software that facilitates 3-D lung tumor segmentation, licensed to Varian Medical Systems, has been incorporated into the Smart Segmentation module of Varian's Eclipse treatment planning system ...

Team develops advanced video and image processing

March 27, 2013

Rapid developments in satellite and sensor technologies have increased the availability of high-resolution, remotely sensed images faster than researchers can process and analyze the data manually.

Recommended for you

Permanent, wireless self-charging system using NIR band

October 8, 2018

As wearable devices are emerging, there are numerous studies on wireless charging systems. Here, a KAIST research team has developed a permanent, wireless self-charging platform for low-power wearable electronics by converting ...

Facebook launches AI video-calling device 'Portal'

October 8, 2018

Facebook on Monday launched a range of AI-powered video-calling devices, a strategic revolution for the social network giant which is aiming for a slice of the smart speaker market that is currently dominated by Amazon and ...

Artificial enzymes convert solar energy into hydrogen gas

October 4, 2018

In a new scientific article, researchers at Uppsala University describe how, using a completely new method, they have synthesised an artificial enzyme that functions in the metabolism of living cells. These enzymes can utilize ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.