Common sense in robots isn't so common, but this Pictionary-like game could help change that

Super Bowl commercials this year featured robots and intelligent assistants interacting with humans in ways that far surpass the capabilities of real-world systems today. In one rather meta advertisement for a telecom provider, robots brainstorm with humans to come up with the premise for another commercial.

At the Allen Institute of Artificial Intelligence, a private research center perched on the north shore of Lake Union in Seattle, computer scientists are working on imbuing software with humanlike abilities to recognize images and understand language that could someday make that sort of collaboration possible.

Their latest effort is a game modeled on the guessing-and-drawing diversion Pictionary, itself a Seattle product. Iconary can be played in collaboration with a collection of algorithms nicknamed AllenAI, capable of deciphering symbols and language, and even deploying something approaching common sense, said Ani Kembhavi, senior research scientist at the institute.

The game is available online and is free to play at iconary.allenai.org/ . Players depict a word or phrase using a selection of icons and symbols such as arrows, musical notes, human figures and buildings (actual human drawings, like those used in Pictionary, confused the system. Human players still draw, but the drawings pull up a menu of symbols like in the an auto-complete feature of text messaging programs). AllenAI guesses at the phrase or individual words within it. The human player can swap roles with AllenAI to guess phrases that the system depicts.

In a demonstration, Kembhavi tried to get AllenAI to guess the phrase "children singing in a classroom." He selected icons for two boys and a girl, a couple of musical notes, a desk and a laptop computer and submitted the scene. AllenAI guessed "people singing at a table," "crowd singing in a room," "audience singing in a restaurant."

"All plausible, on the right track," Kembhavi said. AllenAI "understood that someone is singing somewhere."

He drew again, adding an icon of a white board to try to better depict the classroom part of the phrase. Kembhavi said the back-and-forth between human player and machine is exciting; AllenAI uses all of a player's past drawings and its past guesses to inform its next guess. After another tweak, the system guessed correctly.

"It's collaborative, it's communicative, it's not adversarial," he said.

In recent years, advanced software systems—given the ill-defined label of artificial intelligence (A.I.) - have famously bested humans at chess and Go. These games, however, have rigid and explicit rules and clearly defined winners and losers within their limited contexts.

AlphaGo, DeepBlue and more recent A.I. systems that play large-scale online strategy games such as StarCraft represent remarkable achievements, Kembhavi said, but show the limits of these systems as much as their potential. There is little of the real-world's ambiguity and nuance in a given chess position.

"The algorithms that work, they're quite intellectually stimulating, but they cannot be picked up and used on a robot or A.I. agent for a real-world application," Kembhavi said.

They're also adversarial, zero-sum exercises pitting people against machines, evoking fictional depictions of killer robots and A.I. systems that subjugate humanity.

In contrast, a player works with the Iconary A.I. system to accomplish a task. That's an attribute of the system the researchers are excited to build on as more people play the game, providing feedback and data to improve it. Their goal is a "collaborative experience so natural and compelling you wouldn't know there wasn't a human on the other end," writes Kembhavi, collaborator Carissa Schoenick and Allen Institute head Oren Etzioni in a blog post.

Pictionary, the game on which Iconary is based, was created in Seattle in 1985 by Robert Angel, who was waiting tables at the time. Tens of millions of copies have been sold and the game is still on the market today, now owned by Mattel.

The visual and linguistic aspects of Pictionary make it "a compelling sandbox for our experiments," Kembhavi said. A good Pictionary player, he said, has many of the attributes—communication, common sense and the ability to complete tasks—that you would want in a digital assistant or robot.

AllenAI merges linguistic and visual skills in the context of the game, with a dose of what Kembhavi terms "common sense reasoning." That's a very human concept, approximated in this system by word associations. For example, the word "dinner" is a meal associated with evening, which is associated with the moon. So AllenAI might guess "dinner" for a depiction of a meal and a moon.

The system employs natural language understanding to associate words in human speech with their various meanings and connotations. Such systems "learn" word meanings by processing large bodies of text—often annotated by humans for the purpose of training algorithms—and parsing attributes such as sentence structure and word patterns.

Kembhavi sees AllenAI as a step on the path toward a more general artificial intelligence—which is one of the goals Paul Allen set out for the institute when he launched it in 2013. It now employs more than 100 people.

"We're trying to come up with a system that can learn how to use common sense knowledge in effective ways," he said.

The name AllenAI is an homage to Paul Allen, selected after his death last year. "We were all obviously very affected by the news," Kembhavi said, adding that he believes Allen would have been "quite excited with this progress."

©2019 The Seattle Times
Distributed by Tribune Content Agency, LLC.