Framework provides new tools to systematically build cooperation; scientists extend theory of repeated games

July 4, 2018, Harvard University
A social dilemma with a variable resource: If the lumberjacks work together to plant as well as fell trees, the natural resource remains valuable. If they are selfish, and log without replanting, the forest is ruined, earning them less or no money. Only a great deal of cooperative effort can get the forest back to its original, valuable state. Credit: IST Austria/Birgit Rieger

All it took for rewriting the rules of understanding evolution of cooperation was a series of chance encounters between Martin Nowak, Krishnendu Chatterjee and Christian Hilbe.

Chatterjee, a computer science professor at IST Austria, mentioned the idea of stochastic games—games which can change based on players' actions—during his first visit to Harvard in 2008, and the idea sent Nowak down a years-long path to merge the concept with evolutionary dynamics.

"People who study evolution of cooperation do not use stochastic games," said Nowak, who developed the new framework in collaboration with Chatterjee, Hilbe, a post-doctoral fellow in Chatterjee's group at IST, and Stepan Simsa of the Charles University in Prague. "Instead in a sequence of repeated encounters, it is assumed that the same game with the same payoff matrix is played again and again. In a stochastic game, the game itself can change probabilistically depending on the player's actions."

That new approach, described in a July 4 paper published in Nature, describes a system that can model the evolution of cooperation based on repeated stochastic games. In the simplest form only a pair of games is used—one with a larger reward and the other with a smaller reward—in which players make a choice of whether to cooperate or to defect.

Both games work in the same way: If one player defects while the other cooperates, the defector collects a larger reward, while the other person gets nothing. If both defect, both receive a reward, albeit one that is smaller than the reward for cooperation. Under normal conditions, cooperation rarely emerges in such games, because the most logical response is for players to defect in an effort to maximize their reward.

The innovation is that whether or not players cooperate affects which game they subsequently play.

At the start, the players begin with the higher-value game. As long as both players cooperate, they continue to play that game, but a defection from either player results in them moving to the lower-value game. Once both players again cooperate, they might return to the higher-value game.

"Amazingly even if both games are set up so that cooperation does not evolve, when we put them together, we get cooperation," Nowak said. "It's almost like a paradox."

The key to making the system work, Nowak said, is the difference in value between the two games.

"If we defect, we are destroying something, but if we cooperate we are building something," he said. "So when we cooperate, we play subsequently for something that is more valuable, and if we defect, we play subsequently for something that is less valuable.

"That's what makes the new approach exciting," Nowak added. "The idea is so simple and yet it changes everything. If you defect in the first game, you lose out twice, because your opponent will retaliate and you have to play a less valuable game."

The stochastic framework can also be applied to multi-player games, including public goods games and there resolve what is known as the "Tragedy of the commons".

"Humans are exploiting the environment in a public goods game," Nowak said. "In the old framework, we decide to cooperate or defect, but the next day we play the same game again, and the state of the environment is always the same. But in our new theory, if we exploit the environment badly, in the next round it may be deteriorated, and then we face a less valuable public good," he added.

If the environment deteriorates sharply in response to defection, then there is a strong incentive to maintain cooperation. If the environment deteriorates slowly or not at all, then achieving cooperation is—paradoxically -more difficult.

"This has interesting implications for some of the major problems humans are facing, such as climate change, environmental destruction and migration" Nowak said. "If people understand that defection today means that we will play a game with a lower payoff tomorrow, then cooperation becomes a winning strategy."

If we realize that we are at a breaking point, there is a strong rational to cooperate.

The concept can also be used by public officials and policy makers to design programs that empower cooperation. If players cooperate they have the chance to move to more valuable games.

Explore further: Study examines how 'partner and rival' strategies can foster or destroy cooperation

More information: Christian Hilbe et al, Evolution of cooperation in stochastic games, Nature (2018). DOI: 10.1038/s41586-018-0277-x

Related Stories

The logic of modesty—why it pays to be humble

May 28, 2018

Why do people make anonymous donations, and why does the public perceive this as admirable? Why do we downplay our interest in a potential partner if we risk missing out on a relationship? A team of scientists, consisting ...

Evolution of cooperation through longer memory

April 19, 2017

When we make a decision about whether or not to cooperate with someone, we usually base our decision on past experiences—how has this person behaved in the past?—and on future reciprocity—will they return the favor?—and ...

Punishment does not earn rewards or cooperation, study finds

March 19, 2008

Individuals who engage in costly punishment do not benefit from their behavior, according to a new study published this week in the journal Nature by researchers at Harvard University and the Stockholm School of Economics.

Solving the puzzle of cooperation in group environments

November 13, 2014

Research has shown that when two individuals meet repeatedly they are more likely to cooperate with one another. Flávio Pinheiro and colleagues from the Universities of Minho and Lisbon show that the most successful strategy ...

Recommended for you

Solving the mystery of an unusual medieval text

July 20, 2018

When historian Rowan Dorin first stepped onto the Stanford campus in early 2017, he made it a habit to visit Green Library every week to dig through its collection of medieval documents and objects.


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.