You might not have noticed, but over the weekend a little coup took place. On Friday night, in front of a crowd of thousands, an AI bot beat a professional human player at Dota 2 one of the worlds most popular video games. The human champ, the affable Danil \"Dendi\" Ishutin, threw in the towel after being killed three times, saying he couldnt beat the unstoppable bot. It feels a little bit like human, said Dendi. But at the same time, its something else. <\/p>\n
The bots patron was none other than tech billionaire Elon Musk, who helped found and fund the institution that designed it, OpenAI. Musk wasnt present, but made his feelings known on Twitter, saying: OpenAI first ever to defeat world's best players in competitive eSports. Vastly more complex than traditional board games like chess & Go. Even more exciting, said OpenAI, was that the AI had taught itself everything it knew. It learned purely by playing successive versions of itself, amassing lifetimes of in-game experience over the course of just two weeks. <\/p>\n
But how big a deal is all this? Was Friday nights showdown really more impressive than Googles AI victories at the board game Go? The short answer is probably not, but it still represents a significant step forward both for the world of e-sports and the world of artificial intelligence. <\/p>\n
First, we need to look at Musks claim that Dota is vastly more complex than traditional board games like chess & Go. This is completely true. Real-time battle and strategy games like Dota and Starcraft II pose major challenges that computers just cant handle yet. Not only do these games demand long-term strategic thinking, but unlike board games they keep vital information hidden from players. You can see everything thats happening on a chess board, but you cant in a video game. This means you have to predict and preempt what your opponent will do. It takes imagination and intuition. <\/p>\n
In Dota, this complexity is increased as human players are asked to work together in teams of five, coordinating strategies that will change on the fly based on which characters players choose. To make things even more complex, there are more than 100 different characters in-game, each with their own unique skill set; and characters can be equipped with a number of unique items, each of which can be game-winning if deployed at the right moment. All this means its basically impossible to comprehensively program winning strategies into a Dota bot. <\/p>\n
But, the game that OpenAIs bot played was nowhere near as complex as all this. Instead of 5v5, it took on humans at 1v1; and instead of choosing a character, both human and computer were limited to the same hero a fellow named the Shadow Fiend, who has a pretty straightforward set of attacks. My colleague Vlad Savov, a confirmed Dota addict who also wrote up his thoughts on Fridays match, said the 1v1 match represents only a fraction of the complexity of the full team contest. So: probably not as complex as Go. <\/p>\n
The second major caveat is knowing what advantages OpenAIs agent had over its human opponents. One of the major points of discussion in the AI community was whether or not the bot had access to Dotas bot API which would let it tap directly into streams of information from the game, like the distances between players. OpenAIs Greg Brockman confirmed to The Verge that the AI did indeed use the API, and that certain techniques were hardcoded in the agent, including the items it should use in the game. It was also taught certain strategies (like one called creep block) using a trial-and-error technique known as reinforcement learning. Basically, it did get a little coaching. <\/p>\n
Andreas Theodorou, a games AI researcher at the University of Bath and an experienced Dota player, explains why this makes a difference. One of the main things in Dota is that you need to calculate distances to know how far some [attacks] travel, he says. The API allows bots to have specific indications of range. So you can say, If someone is in 500 meters range, do that, but the human player has to calculate it themselves, learning through trial and error. It really gives them an advantage if they have access to information that a human player does not. This is particularly true in a 1v1 setting with a hero like Shadow Fiend; where players have to focus on timing their attacks correctly, rather than overall strategy. <\/p>\n
Brockmans response is that this sort of skill is trivial for an AI to learn, and was never the focus of OpenAIs research. He says the institutes bot could have done without information from the API, but youd just be spending a lot more of your time learning to do vision, which we already know works, so whats the benefit? <\/p>\n
So, knowing all this, should we dismiss OpenAIs victory? Not at all, says Brockman. He points out that, perhaps more important than the bots victory, was how it taught itself in the first place. While previous AI champions like AlphaGo have learned how to play games by soaking up past matches by human champions, OpenAIs bot taught itself (nearly) everything it knows. <\/p>\n
You have this system that has just played against itself, and it has learned robust enough strategies to beat the top pros. Thats not something you should take for granted, says Brockman. And its a big question for any machine learning system: how does complexity get into the model? Where does it come from? <\/p>\n
As OpenAIs Dota bot shows, he says, we dont have to teach computers complexity: they can learn it themselves. And although some of the bots behavior was preprogrammed, it did develop some strategies by itself. For example, it learned how to fake out its opponents by pretending to trigger an attack, only to cancel at the last second, leaving the human player to dodge an attack that never comes exactly like a feint in boxing. <\/p>\n
Others, though, are still a little skeptical. AI researcher Denny Britz, who wrote a popular blog post that put the victory in context, tells The Verge that its difficult to judge the scale of this achievement without knowing more technical details. (Brockman says these are forthcoming, but couldnt give an exact time frame.) Its not clear what the technical contribution is at this point before the paper comes out, says Britz. <\/p>\n
Theodorou points out that although OpenAIs bot beat Dendi onstage, once players got a good look at its tactics, they were able to outwit it. If you look at the strategies they used, they played outside the box a bit and they won, he says. The players used offbeat strategies the sort that wouldnt faze a human opponent, but which the AI had never seen before. It didnt look like the bot was flexible enough, says Theodorou. (Brockman counters that once the bot learned these strategies, it wouldnt fall for them twice.) <\/p>\n
All the experts agree that this was a major achievement, but that the real challenge is yet to come. That will be a 5v5 match, where OpenAIs agents have to manage not just a duel in the middle of the map, but a sprawling, chaotic battlefield, with multiple heroes, dozens of support units, and unexpected twists. Brockman says that OpenAI is currently targeting next years grand Dota tournament in 12 months time to pull this off. Between now and then, theres much more training to be done. <\/p>\n
<\/p>\n
See more here:<\/p>\n
Did Elon Musk's AI champ destroy humans at video games? It's complicated - The Verge<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" You might not have noticed, but over the weekend a little coup took place. On Friday night, in front of a crowd of thousands, an AI bot beat a professional human player at Dota 2 one of the worlds most popular video games. The human champ, the affable Danil \"Dendi\" Ishutin, threw in the towel after being killed three times, saying he couldnt beat the unstoppable bot. Continue reading