The underlying mission of AlphaGo and DeepMind

In an interview with Nature magazine, Demis Hassabis -Google’s DeepMind’s prodigious CEO- said that the big challenge for society over the next decade or two is dealing with the large amounts of data being collected -meteorological, healthcare, economic, etc.- and make sense of that data by finding its underlying structure and make insights on it. For that reason DeepMind has developed an algorithm that can play Go, and recently it did so well that it beat the current world champion Lee Sedol in the first of five projected games.

Note: AlphaGo won 9-dan pro Go master Lee Sedol four games to one.

But why Go and why even play games?

DeepMind is a tech startup acquired by Google (the company’s largest European acquisition) in a series of acquisitions by big companies such as Apple and IBM of startups with Deep learning expertise. Deep learning is a neural network, which is a system inspired by the interconnectivity of neurons in the brain that mimic human learning by changing the strength of simulated neural connections based on experience. It receives raw data and progressively abstracts features from it to output predictions on that data.

Understanding the mind or the world is an incredibly difficult problem, but games allow one to break this down into parts that can be studied. Games are to AI a stripped-back system in which to test theories about the mind, they are a microcosm of the outside world.

Google DeepMind has previously developed a neural network that has learned to play on its own a series of 42 arcade games like Space Invaders (actually spelled Spaaaaaace Invaders). To do this they combined two different machine-learning methods: 1. Deep learning, a brain-inspired architecture in which connections between layers of simulated neurons are strengthened on the basis of experience, and 2. Reinforcement learning, a decision-making system inspired by the dopamine reward system in the brain, learning by trial and error which actions to take at any given time to bring the greatest rewards.

The breakthrough from previous AI boardgame winners like IBM’s Deep Blue, which beat chess world champion Garry Kasparov in 1997, and the recently unveiled algorithm that plays Texas Hold ‘Em poker is that those were explicitly programmed to win at those specific games. But DeepMind (the algorithm preceding AlphaGo) was not preprogrammed to play arcade games, it used general algorithms. This means that similar techniques could be applied to other AI domains that require recognition of complex patterns, long-term planning and decision-making.

AlphaGo was built on those algorithms to play Go, combining an advanced tree search with deep neural networks containing millions of neuron-like connections. It learned to discover new strategies for itself by playing thousands of games between its neural networks, reinforcing its connections by trial-and-error.

Furthermore Go has been the only remaining board game harder than chess to be tackled by AI. It has long been viewed as one of the greatest challenges for AI for its toll on computing power. Chess has 20 average moves for every position, but Go has 200. The number of possible board configurations is larger than the number of atoms in the universe. For that reason it depends on intuition much more than Chess, which means that constructing a search tree over all possible positions is not an option and more elegant measures needed to be taken. Hassabis described it as “the deepest and most profound game that man has ever devised.”

We are still far from applying DeepMind and AlphaGo’s insights into other realms of AI, Hassabis said, but the benchmarks have been set and a line of approach in the quest for AI is being paved.

Note: As a closing statement after the last of five games (6:56:40 of the feed), Hassabis stated: “In developing AlphaGo we have created some general purpose algorithms that we think one day can be used in all sorts of real world problems, from healthcare to science. And we hope to explore that in the next few years, and extend the techniques that we’ve developed for AlphaGo. So of course this technologies hold great promise but it is still early days and there’s much work still to be done. As with all powerful technologies they bring opportunities and challenges, and we have to make sure as developers of this kinds of systems and all AI research around the world to think about the ethical responsibilities to build this systems in the right way and to deploy them for the right purposes.”

Note: a previous version of this post mentioned that AlphaGo wasn’t preprogrammed to specifically play Go, but learned on its own like DeepMind. AlphaGo was programmed to play Go but learned new strategies on its own based general algorithms. Commenters corrected me.

References:

Hassabis, Demis. “AlphaGo: Using Machine Learning to Master the Ancient Game of Go.” Official Google Blog. Google DeepMind, 27 Jan. 2016. https://googleblog.blogspot.co.il/2016/01/alphago-machine-learning-game-go.html

Gibney, Elizabeth. “Game-playing Software Holds Lessons for Neuroscience.” Nature.com. Nature Publishing Group, 25 Feb. 2016.

Gibney, Elizabeth. “Google AI Algorithm Masters Ancient Game of Go.” Nature.com. Nature Publishing Group, 27 Jan. 2016.

“Match 5 - Google DeepMind Challenge Match: Lee Sedol vs AlphaGo.” YouTube. 15 Mar. 2016.

The underlying mission of AlphaGo and DeepMind was originally published on LinkedIn.