Software automatically transforms movie clips into comic strips

Mar 14, 2012 by Lisa Zyga feature
A comic page generated from a movie clip of “Titanic.” Image credit: Wang, et al. ©2012 IEEE

(PhysOrg.com) -- While some comics today are still drawn by hand, many modern cartoonists use a variety of digital tools to create comics. But even with the help of these tools, creating comics is a time-consuming task that requires many human hours of work. In a new study, a team of researchers has designed a program that can automatically transform movie scenes into comic strips, without the need for any human intervention.

Meng Wang, a professor at the Hefei University of Technology, , and coauthors have published their study on the new software, which they call “Movie2Comics,” in a recent issue of IEEE Transactions on Multimedia.

“The software can be useful for both professionals and hobbyists,” Wang told PhysOrg.com. “Professionals can directly use the software to generate (or integrate their interaction to achieve more impressive results); hobbyists may have interests to try it to see what will be generated from different movie clips.”

As the researchers explain in their study, previous programs have been developed to assist cartoonists in converting movies into comics, but the new method is the first fully automated approach. Without the need for any manual intervention, the method has the potential to significantly cut the time and expense associated with creating comics.

The new cartoonization process involves several steps, including an automatic script-face mapping algorithm that identifies the speaking character in scenes with multiple characters, automatic generation of comic panels of different sizes, positioning word balloons, and rendering movie frames in a cartoon style.

The researchers used the new method to transform 15 movie clips into . The clips came from three movies – “Titanic,” “Sherlock Holmes,” and “The Message” – and varied in length from 2 to 7 minutes. Although the method performed the transformation well for the most part, it sometimes put word bubbles next to the faces of incorrect characters. The script-face mapping algorithm had an accuracy of 85%, which the researchers hope to improve.

A comic page generated from a movie clip of “The Message.” Image credit: Wang, et al. ©2012 IEEE

The researchers then conducted a user evaluation to see how well users understood and enjoyed the comics. They found that users had a slightly lower content comprehension of the comics compared with the original movie clips. Although some loss in comprehension is inevitable when converting to a different format, the script-face mapping errors also played a role. As for enjoyment, the user evaluations showed that enjoyment was highest when the comics included word balloons and effective layout and stylization.

Although the technique is capable of performing all steps automatically, the researchers noted that involving some human effort could lead to even better results. In such a scenario, the software would provide recommendations for each step of the transformation process, and humans could manually adjust the results much more quickly and efficiently than in pure manual methods. But the researchers also hope to improve the accuracy of the automated method.

“We have two future plans,” Wang said. “One, improve the performance of each component, such as script-face mapping, and hope we can generate perfect clips without user interaction. Two, integrate speech recognition technology to generalize the software, such that we can generate comics without movie scripts (currently we need the script file of the movie clip, but by using speech recognition we can automatically transform speech to texts).”

Explore further: Avatars make the Internet sign to deaf people

More information: Meng Wang, et al. “Movie2Comics: Towards a Lively Video Content Presentation.” IEEE Transactions on Multimedia. DOI: 10.1109/TMM.2012.2187181

2.9 /5 (15 votes)

Related Stories

PHD Comics hits the big screen

Sep 16, 2011

(PhysOrg.com) -- If you are a graduate student, you are more than likely aware of the popular Piled Higher and Deeper, or PHD, Comics created by Jorge Cham. These comics cover the everyday struggles that s ...

It's a bird, it's a plane, it's super hero sexism

Dec 21, 2011

As parents do their final holiday shopping, comic books, and their related superhero-themed toys and children's gear, continue to be popular. From Batman rain boots and Legos, to paperback books about Wonder Woman, many stores ...

Comics are serious teaching tools for USC linguist

Jan 10, 2012

The study of linguistics is not a laughing matter -- unless you happen to have Stan Dubinsky as your professor. The University of South Carolina linguist has been sharing jokes and puns and cartoons with students ...

Study: 'Superwomen' in movies affect real women

Jul 20, 2011

An Angelina Jolie character who crushes a robot in “Tomb Raider” is perceived by young women as a better role model than a mouthy Kathy Bates character who carries a gun in “Primary Colors,” a UC Davis ...

Researchers hope to build universal human age estimator

Dec 05, 2011

(PhysOrg.com) -- As humans, we have a knack for estimating another person’s age quite accurately just by glancing at their face. Although age estimation may seem relatively simple to us, computers have ...

Recommended for you

Avatars make the Internet sign to deaf people

Aug 29, 2014

It is challenging for deaf people to learn a sound-based language, since they are physically not able to hear those sounds. Hence, most of them struggle with written language as well as with text reading ...

Chameleon: Cloud computing for computer science

Aug 26, 2014

Cloud computing has changed the way we work, the way we communicate online, even the way we relax at night with a movie. But even as "the cloud" starts to cross over into popular parlance, the full potential ...

User comments : 7

Adjust slider to filter visible comments by rank

Display comments: newest first

drel
5 / 5 (7) Mar 14, 2012
Hand drawn cartoons are art created by artists. This looks like crap. Degenerated photos are not cartoons. The technology may have its uses, but as comics it's a FAIL.
Isaacsname
not rated yet Mar 14, 2012
Somebody do the State of the Union speech,... please.
RazorsEdge
not rated yet Mar 17, 2012
Somebody do the State of the Union speech,... please.


The software has to replicate the speech patterns like "aaaannnnd". That would be funny.
Callippo
1 / 5 (1) Mar 17, 2012
I found the watching of many movies a waste of time and reading of subtitle files counterintuitive to the actual content of movies. Background image is worth of thousands words after then. This technology could become useful for automatic generation of excerpts from movies.
neiorah
not rated yet Mar 19, 2012
The better not mess with the fantastic artwork of modern anime.
Vendicar_Decarian
not rated yet Mar 20, 2012
The reverse would be much more interesting.
art_kirakosyanc
not rated yet Mar 21, 2012
Hi i think the invention is very useful, and think that the guys do great job, don't critique instantly (to others), i see here a huge business opportunity. Please tell how i can contact with the creator of this software and is there any prototype put online?
Thanks in advance