Beautiful brushstrokes drawn from data

September 26, 2013 by John Sullivan, Princeton University
The graphics program RealBrush, which was developed by a team of researchers including Princeton scientists, can not only create authentic brushstrokes in a wide variety of materials, it can also exactly render effects such as smudging and overlapping. Credit: Jingwan Lu et al.

( —A good painter uses simple strokes of a brush to bring texture, contrast and depth to a blank canvas. In comparison, computer programs can have difficulty reproducing the complex and varied forms of brushstrokes, and often require painstaking effort to mimic a brief sweep of paint.

Now, a team of researchers including scientists at Princeton University has developed a program that allows to quickly and easily produce realistic brushstrokes on their computers. Called RealBrush, the program combines graphics algorithms with "Big Data" storage and retrieval techniques to allow computer artists to create, bend and shape a wide array of brushstrokes. Besides creating the strokes, the program allows for effects including smudging, smearing and merging of different types of media.

"Our goal is to have it look like a photograph of a real stroke but to have it follow whatever path you happen to be drawing," said Adam Finkelstein, a professor and the senior researcher on the project. "The challenge is if you have a photo of a paint stroke, how do you make it fit a different path?"

As a first step, a RealBrush user creates a few sample strokes using any desired medium, from oil paint to nail polish or even toothpaste. The user then photographs the strokes and enters them into RealBrush's library (the program also has a variety of pre-installed strokes in different media.) The program uses the sample brushstrokes as baselines indicating of the strokes—for instance, the program looks at the center line, or spine, of a segment to help determine a basic shape. The program can use those samples to warp and blend the original strokes into any curves or forms the user desires.

"We use feature matching—it is a machine-learning approach," said Jingwan "Cynthia" Lu, a graduate student in computer science who was the lead author of a paper the researchers presented at the SIGGRAPH computer graphics conference in Anaheim, Cal., in August. "Based on the shape of the spine, we want to find segments of the exemplar that have a similar spine and use the segments to make similar shapes."

In this video, Lu discusses her research and how it allows her to bring her interests in various art forms to her work:

RealBrush uses a comparable approach to simulate other graphic effects: "shape," the silhouette and texture of a single stroke; "smear," a new stroke applied across wet paint, which causes a mingling of colors; and "smudge," the effect of a dry brush or finger rubbed across a brushstroke. The researchers left one more feature for future work—"composite," which is the glazing or reflective effect of translucent paints layered on top of each other.

"For each style, we need about 20 to 30 strokes," Lu said. "If you are a casual user, you can use pre-captured strokes. Or you can paint your own strokes and record those in our system. You can share those with your friends."

Lu worked out the basics of RealBrush during an internship working with a senior researcher, Stephen DiVerdi, at Adobe Systems in San Francisco. As part of the project, she collaborated with Finkelstein; Connelly Barnes, a Princeton 2011 Ph.D. recipient now transitioning from a postdoc at Adobe to a faculty position in computer science at the University of Virginia; and DiVerdi, who is now at Google, to create the paper that was presented at SIGGRAPH. Although RealBrush is a working prototype, Finkelstein said it is a research project, and he noted that it would take additional effort before it could be available for widespread use.

Finkelstein said the RealBrush approach is an example of a change in programming that has the potential for huge impact in many fields, including graphic design.

"In the old days, people wrote clever algorithms to simulate what they were trying to achieve," he said. "What algorithm can I use that will look like a brush on canvas? It was hard, and it met with limited success."

Now, he said, graphics programming is "entering the world of Big Data."

Big Data—a term for the paradigm shift in which the ability to capture, store and analyze huge quantities of digital information allows scientists to leverage this data in addressing new challenges—is fundamentally changing many fields of computing.

A Big Data approach allows programmers to use that data to answer questions, or create simulations, through examples rather than pure mathematics. To do that, scientists need to use statistical and machine-learning approaches to draw the answers from the vast amounts of data available.

"How do you draw the fundamental characteristics of paint from many examples of paint? How do you extrapolate traffic patterns from what traffic was like today, or last week, or last month?" Finkelstein said. "We are entering a realm where you don't have to apply a clever algorithm to simulate the world; you can say, 'this is how the world really is.'"

Explore further: Calculating art: Meet e-David, the painting machine (w/ Video)

Related Stories

Young people at higher risk for stroke

September 3, 2013

Fifteen percent of the most common type of strokes occur in adolescents and young adults, and more young people are showing risk factors for such strokes, according to a report in the journal Neurology.

Recommended for you

What do you get when you cross an airplane with a submarine?

February 15, 2018

Researchers from North Carolina State University have developed the first unmanned, fixed-wing aircraft that is capable of traveling both through the air and under the water – transitioning repeatedly between sky and sea. ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.