When artificial intelligence evaluates chess champions

April 25, 2017 by Alexiane Agullo, CNRS
When artificial intelligence evaluates chess champions
Predictions of game results between the different world champions in their best year. Credit: Jean-Marc Alliot – IRIT (CNRS/INP Toulouse/Université Toulouse Paul Sabatier/Université Toulouse Jean Jaurès/Université Toulouse Capitole)

The ELO system, which most chess federations use today, ranks players by the results of their games. Although simple and efficient, it overlooks relevant criteria such as the quality of the moves players actually make. To overcome these limitations, Jean-Marc Alliot of the Institut de recherche en informatique de Toulouse (IRIT - CNRS/INP Toulouse/Université Toulouse Paul Sabatier/Université Toulouse Jean Jaurès/Université Toulouse Capitole) demonstrates a new system, published on 24 April 2017 in the International Computer Games Association Journal.

Since the 1970s, the system designed by the Hungarian, Arpad Elo, has been ranking chess according to the result of their games. The best players have the highest ranking, and the difference in ELO points between two players predicts the probability of either player winning a given game. If players perform better or worse than predicted, they either gain or lose points accordingly. This method does not take into account the quality of the moves played during a and is therefore unable to reliably rank players who have played at different periods. Jean-Marc Alliot suggests a direct ranking of players based on the quality of their actual moves.

His system computes the difference between the move actually played and the move that would have been selected by the best chess program to date, Stockfish. Running on the OSIRIM supercomputer, this program executes almost perfect moves. Starting with those of Wilhelm Steinitz (1836-1900), all 26,000 games played since then by chess world champions have been processed in order to create a probabilistic model for each player. For each position, the model estimates the probability of making a mistake, and the magnitude of the mistake. These models can then be used to compute the win/draw/lose probability for any given match between two players. The predictions have proven not only to be extremely close to the actual results when players have played concrete games against one another, they also fare better than those based on ELO scores. The results demonstrate that the level of players has been steadily increasing. The current world champion, Magnus Carlsen, tops the list, while Bobby Fischer is third.

Under current conditions, this new ranking method cannot immediately replace the ELO system, which is easier to set up and implement. However, increases in computing power will make it possible to extend the new method to an ever-growing pool of players in the near future.

Explore further: Researchers analyze chess behavior

More information: Who is the Master? Jean-Marc Alliot, ICGA Journal, Volume 39-1, Avril 2017. DOI: 10.3233/ICG-160012 , content.iospress.com/articles/icga-journal/icg0012

Related Stories

Researchers analyze chess behavior

December 23, 2016

Chess is one of the oldest—and most popular—board games. On Christmas Eve, the classic game is given as a gift several hundred thousand times over, whether as a chess set, computer game, or chess computer. Yet what is ...

How computers changed chess

November 29, 2013

For his precocity, newly crowned World Chess Champion Magnus Carlsen was named "The Mozart of chess". For his tenacity and comfort with long games he was dubbed "The Nadal of chess". More prosaically, the BBC called him "The ...

Know when to fold 'em: AI beats world's top poker players

January 31, 2017

If you were about to start playing a game of online poker, you might want to think again. Humankind has just been beaten at yet another game, this time Heads-Up No-Limit Texas Hold'em poker. This is a milestone moment for ...

Predicting future sports rankings from evolving performance

December 20, 2016

Competitive sports and games are all about the performance of players and teams, which results in performance-based hierarchies. Because such performance is measurable and is the result of varied rules, sports and games are ...

Recommended for you

Team breaks world record for fast, accurate AI training

November 7, 2018

Researchers at Hong Kong Baptist University (HKBU) have partnered with a team from Tencent Machine Learning to create a new technique for training artificial intelligence (AI) machines faster than ever before while maintaining ...

1 comment

Adjust slider to filter visible comments by rank

Display comments: newest first

antialias_physorg
not rated yet Apr 25, 2017
I wonder if this is a fair way to rate players. In play (any game) it's notimportant to make the best possible move. It's only important to make better moves than the opponent.

In some games making a 'best possible' move can even be counterproductive. E.g. in chess or poker there are moves where you try to trap of bluff an opponent - but if your opponent isn't good enough to realize what you are trying to represent (e.g. a false opportunity for them to make a move) then making such a move is a waste and potentially puts you at a disadvantage against someone who performs simpl/straight-forward play.

In the end: If you're the best player you can only be as good as your best opponent demands it of you.

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more

Click here to reset your password.
Sign in to get notified via email when new comments are made.