The Human Checkmate
Notice: Undefined index: gated in /opt/bitnami/apps/wordpress/htdocs/wp-content/themes/thenewatlantis/template-parts/cards/25wide.php on line 27
Last August, two dozen of the strongest chess players in the world met for a new kind of tournament. The ground rules were unforgiving: Each player would start with only fifteen minutes on the clock per game, competing in scores of back-to-back games against all the others multiple times. Players were eliminated in stages until only two remained, who would go on to play two hundred games to decide the champion. Competitors hailed from around the world, though a sizeable minority were American. All would face the constant scrutiny of fans through an online broadcast of the tournament on Twitch, a streaming website. A month of grueling play later, a victor emerged: Stockfish 220818, the strongest chess computer to date.
While many past computer chess competitions had the human programmers convene in person, in the Computer Chess Championship, teams submitted their software to run on the servers of Chess.com, which hosted the event. The Twitch feed showed not only the live game play, but “a real-time peek into” each program’s “thinking process and the lines they are considering,” said a post announcing the tournament. The website ran a single game at a time, back to back, with uninterrupted play for a month.
In November, while the chess universe was watching the official, human championship match in London, won by Magnus Carlsen of Norway, the computers — the game’s real elite — were still playing in their own continuous series of online tournaments.
Chess computers, also known as chess engines, have different personalities on the board. Stockfish, an open-source engine freely available and maintained by a community of programmers, has a clean, positional style, Pete Cilento, executive editor at Chess.com, told me by email. Leela Chess Zero, also known as Lc0, another open-source engine, “plays a more intuitive and hazy game,” he said. “It has gained so many fans because it plays superhuman chess in a human way.” Houdini, developed by programmer Robert Houdart, has a more aggressive and sacrificial style, which is why it has been compared to the great players of the Romantic Era. In a way, they are human: Behind every engine is a team of programmers, engineers, and chess experts.
Decades before self-driving cars or Siri, chess was an obsession of AI researchers, and getting a computer to beat a human master their holy grail. Today, twenty-two years after IBM’s Deep Blue shocked the world by beating then–world champion Garry Kasparov, chess computers have left humans in the dust. The latest generation of programs, such as AlphaZero, developed by the Alphabet-owned company DeepMind, is doing things that even their human creators don’t understand. Demis Hassabis, co-founder and CEO of DeepMind, has described aspects of AlphaZero’s decision-making as a “black box,” for example, how it assesses the overall value of a rook compared to a knight. “We don’t actually know.”
The impact of computer chess on the game — as still played by humans — has been twofold. First, computers have helped to flatten chess, increasing pure understanding of the game at the expense of creativity, mystery, and dynamism. Second, they have become intertwined with every aspect of chess, from play at the highest level to amateur study and the spectator’s experience. These two effects mirror how emerging technologies are changing the way we live. Chess today is a window into the future, when machine learning is applied to all kinds of human endeavors.
Unlike poker — or even baseball, where a gust of wind can push a fair ball foul — chess is a game without chance. Its drama has to come from the creativity and daring of its players. But in part because of computer chess, that daring may be increasingly hard to find in human players.
The first chess-playing machine was anything but boring. The Turk, a cabinet of gears and cogs built around 1770, featured a life-sized figure dressed in Ottoman robes that could mechanically move the chess pieces. “The Chess Player has withstood the first players of Europe and America, and excites universal admiration,” boasted one 1834 advertisement. “He moves his head, eyes, lips, and hands, with the greatest facility.” It traveled the world beating well-regarded players like Benjamin Franklin and Napoleon, but was later revealed as a hoax. Inside the cabinet, a human was pulling the strings.
By 1950, the real thing was more plausible, when computer science pioneer Claude Shannon wrote his landmark paper “Programming a Computer for Playing Chess.” Chess seemed like an ideal game for programming, Shannon thought. Its rules are clearly defined, and it is challenging for humans to play well. It also requires considerable skill and analytic intelligence, which is why building a chess machine “will force us either to admit the possibility of mechanized thinking or to further restrict our concept of ‘thinking.’” Researchers and observers would come to regard the competitiveness of chess computers as a stand-in for the overall state of the field of artificial intelligence.
It was a few decades more before computers actually became useful for human chess competitors. Garry Kasparov, who first became world champion in 1985, was an early adopter. Two years later he used an early version of ChessBase, a database that would allow a player to prepare for a competition by reviewing his upcoming opponent’s previous games. ChessBase helped Kasparov to plan for a match in which he would compete against eight different German players simultaneously. As he writes in his 2017 book Deep Thinking: Where Machine Intelligence Ends and Human Creativity Begins, “With just two days of preparation I felt comfortable going into the match and won in crushing fashion, 7–1. That was when I knew I was going to be spending a lot of time in front of a computer for the rest of my career.”
But chess engines per se — programs that actually play the game — were then not yet good enough for Kasparov. “Even as they got much stronger and became dangerous opponents in the early 1990s, the chess they played was ugly and inhuman, not very useful for serious training.” That, too, would change. Using both databases and chess engines, Kasparov re-invented himself, going through his repertoire and verifying acquired wisdom against cold analysis.
Over the centuries, chess knowledge had been slowly accumulating through trial and error, game by game. Players would memorize frequently played move sequences whose outcomes were well understood. And because human memory is limited, general principles also developed, ranging from the pithy (“a knight on the rim is dim”) to the highly complex. Over hundreds of thousands of games by the top players, a huge body of theory evolved.
With computer engines, that accumulation of human knowledge was put to the test. Engines can “see” much further down the hypothetical line when contemplating a possible move, so many positions that had seemed playable were now shown to be dead in the water. The number of unclear positions — in which both players thought they stood a fighting chance — began to shrink.
Judit Polgár from Hungary ranked eighth at the World Chess Championship in 2005 and was for a time the youngest person to have ever achieved the title of grandmaster, at age 15, beating American legend Bobby Fischer’s record by a few months. “When I was a kid, I was preparing on paper-based material,” she told me in a phone interview. “Later on, in the early 2000s, it was already very clear that if you don’t use the engine for help or advice, you’re going to be falling behind. It was a struggle for me because I’m a very creative player, and with the computer, many times it points out that maybe I have creative ideas, but not necessarily good ones.”
Shrinking the domain of the uncertain is an aim of artificial intelligence more broadly. Predictive analytics is now used everywhere from personalized medicine to the courtroom and counterterrorism, where algorithms can wade through swaths of personal data to identify trends and possible threats. When lives are at stake, that certainty can seem like a plain good, even if it might conflict with other goods. In chess, however — like art, music, and most everything that entertains us — it might be directly contrary to the point, since knowing what happens in advance tends to ruin the fun. The professionals, looking for any competitive advantage, had no choice but to embrace the new understanding that computers made possible; indeed, they sought it out. But at what cost?
In games between the best grandmasters, the first fifteen moves or more may be prepared in advance by the competitors, who arrive armed with prior knowledge of the best move at every likely turn. That preparation has made it harder for elite players to find an edge. The opening phase of the game has become a kind of trench warfare, a fight to avoid losing ground. One result of rigorous preparation and close matches in player skill level is that draws become more likely — a trend that seems to have been on the rise long before chess engines. According to an analysis by data scientist Randy Olsen, draws have become up to three times more frequent since 1850. Although he found a slight decline since about 1990, many chess commentators point to the general truth that better preparation, which computers aid, have made draws more common in games at the top level.
In the 2018 world championship match between Magnus Carlsen (of Norway) and Fabiano Caruana (of the United States), for the first time ever there was not a single win in the regular games; all twelve ended in draws. A tie-breaking round of rapid chess, one of the only ways to guarantee a victor, was needed to determine the winner, Carlsen.
Draws in chess need not be boring; they can thrill the expert as much as a pitcher’s duel in baseball. But for many chess amateurs, a game without a winner is highly unsatisfying, especially because of the ways that engines have changed how fans consume the game.
Broadcasts of prominent games now typically display an engine’s analysis along with the board. If the white pieces have a small advantage, the display might read +0.46; a decisive advantage for the black pieces could be -2.00. A scoreboard gives anyone, no matter what skill level, a read on who’s doing better.
The scoreboard of choice for many watching the recent world championship between Magnus Carlsen and Fabiano Caruana was an online platform built by Norwegian programmer Steinar H. Gunderson. The website showed the live chessboard for virtually all of Carlsen’s tournament games and provided analysis by the Stockfish engine, including which series of moves would be the best and who was more likely to win at any given point.
Chess in Norway has exploded in popularity with the rise of Carlsen, the world’s top-ranked player since 2011. “Chess on TV is made possible in Norway for two reasons: one is that the Norwegian is there, and the other is the computer analysis,” Gunderson told me by phone. “Without the computer analysis, the drama wouldn’t be there in the same way.”
Most casual chess fans can’t hope to grasp much of what goes on in a game between two grandmasters. Computers have helped to democratize the consumption of high-level chess, giving anyone with an Internet connection access to the deepest chess analysis in history. One result is that amateurs, watching games accompanied by analysis from a chess engine, may “know” the best next move better than the players themselves — yet wouldn’t be able to explain the logic behind it. Professionals complain about armchair quarterbacking; it’s easy for viewers at home to criticize Carlsen for missing a move recommended by the engines, but finding that move over the board is another task entirely.
As chess computers have become increasingly intertwined with the game in every way, the effect has been double-edged for amateurs. Chess engines can be an invaluable way for students to check their play for improvement, or they can serve as a crutch. One joy of chess is in the mental workout, stretching the limits of your own understanding.
For professionals, it’s a more complicated story. In many areas of work today, conversations about artificial intelligence often turn to unemployment: How many jobs will AI create, and how many will it kill? But for those making a living in chess, that future is already here, as professional chess is now in large part an exercise in IT management. Elite players are running engines around the clock, directing their focus and comparing their different evaluations. Part of the craft involves deciding when to employ computer analysis and when to stick to your own ideas. “The thing is, you cannot rely too much on it,” Maxime Vachier-Lagrave (of France), currently the world’s sixth-highest rated player, told me by phone. “If you’re used to looking at computer lines, your brain will not switch on when it’s time to play a game.”
As chess engines have improved, players have learned to use them in new ways. Ideas that seemed to violate the general principles of old could be quickly tested on the computer, and shown to ultimately pay off after many moves down the road. “Now if something appears far-fetched, something challenges the human process of thought, we still take it into consideration,” says Vachier-Lagrave.
Garry Kasparov has long promoted what is known as “advanced” or “centaur” chess. Two human opponents are paired with computers they can freely consult throughout the game. These human–computer teams are better than any individual computer playing alone, although the human halves of the world’s best centaur teams aren’t always very good at conventional chess. The man–machine hybrid is a different beast entirely.
Indeed, the centaur model has become a fashionable way of thinking about the future of artificial intelligence generally. In this view, AI will augment, rather than replace, human intelligence. For today’s elite chess players, who have integrated engines into their lives, that already seems to be true.
“At first, the developers were trying to emulate human play,” said Dirk Jan ten Geuzendam, editor-in-chief of New in Chess magazine, in a phone interview. “At some point we reached a level of computer play where it became the other way around. In fact, the humans are trying to follow the thought process of the computers.”
That role reversal is becoming even more pronounced after some recent dramatic developments in chess programming. In 2017, DeepMind’s AlphaZero program demolished Stockfish, previously the strongest chess-playing program, without losing a single game. Out of a hundred games, it won 28 and drew 72. In a rematch last December, out of a thousand games, it won 155 and lost only 6.
“It doesn’t play like a human, but it also doesn’t play like computer engines,” said DeepMind CEO Demis Hassabis at a 2017 conference. “It sort of plays in a third, almost alien, way.” While conventional engines like Stockfish must be programmed with some general principles about what elements of a position are good or bad, AlphaZero was given only the basic rules of chess, with no human preconceptions. In a 2017 paper, the program’s creators explain:
State-of-the-art programs are based on powerful engines that search many millions of positions, leveraging handcrafted domain expertise and sophisticated domain adaptations. AlphaZero is a generic reinforcement learning algorithm — originally devised for the game of Go — that achieved superior results within a few hours, searching a thousand times fewer positions, given no domain knowledge except the rules of chess.
As machine learning in chess programs becomes more advanced — in other words, less human — players hope to further push their understanding of the game. Hassabis, for example, claimed at the conference that he never knew, until looking at how AlphaZero plays, that sacrificing pieces could be done not just for short-term tactical advantage but long-term positional advantage.
The two effects of chess computers — greater understanding at the expense of less uncertainty, and an intertwining of humans and machines in every part of the game — may seem to point in opposite directions. Has chess today become more advanced, or just more “draw-ish”? Are we opening new avenues of knowledge, or losing some essential quality?
The two may go hand in hand. The very dynamism that makes chess so compelling was the same thing that enticed us to try to solve the game, bringing all of our technological know-how to bear on its magic and mystery. The trick is to accept the new with a spirit of the old.
Nearly a century ago, in another era when chess players were worried about too many draws, the game’s best player was José Raúl Capablanca. World champion from 1921 to 1927, the Cuban Capablanca was the antithesis of today’s hyper-prepared player. He taught himself the game by watching his father play, and said that he never studied books on opening variations.
“Chess can never reach its height by following in the path of science,” goes a quote attributed to Capablanca. “Let us, therefore, make a new effort and with the help of our imagination turn the struggle of technique into a battle of ideas.”
Computer chess is not a battle of ideas, but of programming ingenuity and processing power. So how could we heed Capablanca’s warning today? After the draw-fest at the last world championship, some, including Magnus Carlsen, suggested that changes to the classic format were needed. Shortening the time limit for games could lead to more decisive results by encouraging greater risk-taking.
For Bobby Fischer — world champion in the early 1970s and the last American to hold the title — one solution was to re-introduce uncertainty. In a chess variant he created, the pieces are randomized in different positions every game, making it useless for players to memorize long opening sequences in advance.
With next-generation programs like AlphaZero, computer competitions may well become the place where the most “interesting” chess is being played — at least, the most advanced, error-free chess, and the most demanding for humans to understand. But Capablanca’s noble formulation of chess as a “battle of ideas” fundamentally depends on the possibility of human error. When we play chess, we hatch grand plans, take risks, fall into traps, succumb to pressure, psych out opponents, and make bold sacrifices — all without knowing whether any of it will pay off.
During Covid, The New Atlantis has offered an independent alternative. In this unsettled moment, we need your help to continue.