Dartmouth contest shows computers aren't such good poets

May 19, 2016 by By Michael Casey
In this Wednesday, May 18, 2016, photo, Dartmouth College undergraduate Christopher Hogan, left, spins a 10-minute set of music during a competition on campus in Hanover, N.H., where dancers were asked to determine whether the music was generated by humans or a computer algorithm. (AP Photo/Michael Casey)

Computers are pretty good at stocking shelves and operating cars, but are not so great at writing poetry.

Scientists in a Dartmouth College competition reached that conclusion after designing artificial intelligence algorithms that could produce sonnets. Judges compared the results with poems written by humans to see if they could tell the difference.

In every instance, the judges were able to find the sonnet produced by a computer program.

The yearlong competition was a variation of the "Turing Test," named for British computer scientist Alan Turing, who in 1950 proposed an experiment to determine if computers could have humanlike intelligence. Results were announced Wednesday night.

A three-judge panel that included Pulitzer Prize-winning author Louis Menand was asked to read 10 submissions—six produced by humans and four by two different algorithms. The machines were given nouns—including "wave," ''tourist" and "floor"— and programmed to produce a sonnet. The software packages didn't have the flow or narrative of a good poem.

Some also had "idiosyncrasies of syntax and diction, uses of language that were just a little off," Menand said in an email interview.

In this Wednesday, May 18, 2016, photo, Dartmouth College professor Michael Casey, left, undergraduates Jun Oh, center, and Peter J. O'Sullivan prepare music for a competition on campus in Hanover, N.H., where dancers were asked to determine whether the music was generated by humans or a computer algorithm. (AP Photo/Michael Casey)

Competition co-founder Dan Rockmore, a Dartmouth professor, said he was surprised at the computers' poor showing. But he wasn't that surprised.

"The judges were hunting for machines so they are not looking at a Hallmark card and reading the poem inside," he added.

The results of the competition also included a short story segment and one involving computer and human DJs. Computer algorithms were not much better at writing short stories, failing to fool a panel of judges—although one judge was tricked by one story.

The sets of music produce by DJs—played behind a black curtain at Dartmouth's Hood Museum of Art—were more of a challenge with dancers having a difficult time determining who produced the various sets. Two algorithm entries managed to confound about 40 percent of the dancers.

There have been several efforts to create computer programs that can generate music or text, including a novel this year in Japan written by humans and an artificial intelligence program. The Associated Press, for example, is using computer programs to automatically generate business earnings reports.

Michael Casey, a professor who also helped organize the Dartmouth competition, said the results demonstrated the challenges posed to machines when they try to mimic the arts—though he wasn't giving up that one day a computerized Shakespeare would emerge. (Casey is no relation to the writer of this news article.)

In this Wednesday, May 18, 2016, photo, Dartmouth College students dance on campus in Hanover, N.H., to music that was generated either by humans or a computer and they vote on whether humans or a machine spun each set as the results were beamed in real time onto the wall above them. (AP Photo/Michael Casey)
"All the nuance of a story, all the form and precision in a poem, can that be replicated by algorithm? Maybe," Casey said. "By doing this once, we may be able to encourage whoever is out there working on this kind of thing to take part and maybe we will get better algorithms."

However, Sherry Turkle, a professor at Massachusetts Institute of Technology and the author of "Reclaiming Conversation: The Power of Talk in a Digital Age," who did not take part in the contest, questions the whole idea of trying to get a machine to write a poem.

"Poetry needs to come from the experience of human meaning. That is what gives it life," she said in an email.

Rockmore and Casey countered that algorithms already are playing a role in choosing the books people read and the music they listen to. It would be just a matter of time before computers generate the actual content that is read—but rather than a writer, the artist would be the programmer designing the algorithm.

"What if you could write beautiful stories that made people happy at the snap of your fingers," Rockmore said. "That would be a wonderful thing. It wouldn't mean humans weren't writing great things, too. They are both different art forms."

Computer poem: A floor by any other name would still be ...

In a Dartmouth College computer vs. humans contest, specifically how computers compared with people in writing poetry, scientists designed artificial intelligence algorithms to generate Shakespearean sonnets, of 14 lines and a particular meter. In every instance, judges were able to identify the sonnets produced by computer. In this example, the word "floor" was programmed into an algorithm to produce the sonnet.

The dirty rusty wooden dresser drawer.
A couple million people wearing drawers,
Or looking through a lonely oven door,
Flowers covered under marble floors.
And lying sleeping on an open bed.
And I remember having started tripping,
Or any angel hanging overhead,
Without another cup of coffee dripping.
Surrounded by a pretty little sergeant,
Another morning at an early crawl.
And from the other side of my apartment,
An empty room behind the inner wall.
A thousand pictures on the kitchen floor,
Talked about a hundred years or more.

Explore further: Dartmouth contests showcase computer-generated creativity

Related Stories

Looking for art in artificial intelligence

May 3, 2016

Algorithms help us to choose which films to watch, which music to stream and which literature to read. But what if algorithms went beyond their jobs as mediators of human culture and started to create culture themselves?

Mitsuku chatbot has good answers for the Loebner Prize

September 17, 2013

(Phys.org) —A chatbot named Mitsuku has won the Loebner Prize 2013, announced over the weekend, beating out three other contestants for the top prize of a bronze medal and $4,000. Mitsuku's creator is Steve Worswick, Mitsuku's ...

Professor proposes alternative to 'Turing Test'

November 19, 2014

(Phys.org) —A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally called the ...

Recommended for you

Coffee-based colloids for direct solar absorption

March 22, 2019

Solar energy is one of the most promising resources to help reduce fossil fuel consumption and mitigate greenhouse gas emissions to power a sustainable future. Devices presently in use to convert solar energy into thermal ...

EPA adviser is promoting harmful ideas, scientists say

March 22, 2019

The Trump administration's reliance on industry-funded environmental specialists is again coming under fire, this time by researchers who say that Louis Anthony "Tony" Cox Jr., who leads a key Environmental Protection Agency ...

0 comments

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.