Researchers train computer to create games by watching YouTube

June 26, 2015, Georgia Institute of Technology

Georgia Institute of Technology researchers have developed a computing system that views gameplay video from streaming services like YouTube or Twitch, analyzes the footage and then is able to create original new sections of a game.

The team tested their discovery, the first of its kind, with the original Super Mario Brothers, a well-known two-dimensional platformer that will allow the new automatic-level designer to replicate results across similar games.

The system focuses on the gaming terrain (not the playable character) and the positioning between elements on-screen – be it pipes, blocks, coins or Goombas – and it determines the required relationship or level design rule. For example, pipes in the Mario games tend to stick out of the ground, so the system learns this and prevents any pipes from being flush with grassy surfaces. It also prevents "breaks" by using spatial analysis – e.g. no impossibly long jumps for the hero.

"An initial evaluation of our approach indicates an ability to produce level sections that are both playable and close to the original without hand coding any design criteria," says Matthew Guzdial, lead researcher and Ph.D. student in Computer Science at Georgia Tech.

Key to the process is watching the players in action to see where they actually spend most of their time in the game. After recording on-screen locations of sprites, Georgia Tech's algorithms determine what are high-interaction areas – those spots where players spend more time to collect bonus items or master a challenge. The automatic-level designer specifically targets these areas to gain design information. The system is then able to build a new level section, element by element.

Researchers train computer to create games by watching YouTube
"Our system creates a model or template, and it's able to produce level sections that have never been seen before, do not appear random and can be traversed by the player," says Mark Riedl, associate professor of Interactive Computing. "One could say that the system 'studies' the design of Mario levels until it is able to create new playable areas."

The Georgia Tech system output 151 distinct level sections from 17 samples in the original game, controlling for overall playability and style variables. Output increased to 334 level sections as the system lessened the constraints. The new levels can be played easily by porting them into a game engine.

Riedl says this is the first time he is aware of a gameplay video being used to design levels for a Mario game. By applying the technique across a number of different platformer games, a system can theoretically learn genre knowledge, which can be beneficial for procedurally creating games of a given genre. The technique may also extend to other game genres beyond platformers. The researchers next plan to develop full-scale levels and evaluate how gamers interact in those levels compared to the original gameplay videos.

The research, "Toward Game Level Generation from Gameplay Videos," is featured June 22-25 at the Foundations of Digital Games Conference in Pacific Grove, Calif.

Explore further: Mario and his Mushroom Kingdom hopping into theme parks

More information: Toward Game Level Generation from Gameplay Videos, … _gameplay_videos.pdf

Related Stories

Professor proposes alternative to 'Turing Test'

November 19, 2014

( —A Georgia Tech professor is offering an alternative to the celebrated "Turing Test" to determine whether a machine or computer program exhibits human-level intelligence. The Turing Test - originally called the ...

Recommended for you

Researchers engineer a tougher fiber

February 22, 2019

North Carolina State University researchers have developed a fiber that combines the elasticity of rubber with the strength of a metal, resulting in a tougher material that could be incorporated into soft robotics, packaging ...

A quantum magnet with a topological twist

February 22, 2019

Taking their name from an intricate Japanese basket pattern, kagome magnets are thought to have electronic properties that could be valuable for future quantum devices and applications. Theories predict that some electrons ...


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.