Fans of games like Dungeons & Dragons know that the fun comes, in part, from a creative Dungeon Masteran all-powerful narrator who follows a storyline but has free rein to improvise in response to players actions and the fate of the dice.

This kind of spontaneous yet coherent storytelling is extremely difficult for artificial intelligence, even as AI has mastered more constrained board games such as chess and Go. The best text-generating AI programs too often produce confused and disjointed prose. So some researchers view spontaneous storytelling as a good test of progress toward more intelligent machines.

An attempt to build an artificial Dungeon Master offers hope that machines able to improvise a good storyline might be built. In 2018, Lara Martin, a graduate student at Georgia Tech, was seeking a way for AI and a human to work together to develop a narrative and suggested Dungeons & Dragons as a vehicle for the challenge. After a while, it hit me, she says. I go up to my adviser and say We're basically proposing a Dungeon Master, aren't we? He paused for a bit, and said Yeah, I guess we are!

Narratives produced by artificial intelligence offer a guide to where we are in the quest to create machines that are as clever as us. Martin says this would be more challenging than mastering a game like Go or poker because just about anything that can be imagined can happen in a game.

Since 2018, Martin has published work that outlines progress towards the goal of making an AI Dungeon Master. Her approach combines state-of-the-art machine learning algorithms with more old-fashioned rule-based features. Together this lets an AI system dream up different narratives while following the thread of a story consistently.

Martins latest work, presented at a conference held this month by the Association for the Advancement of Artificial Intelligence, describes a way for an algorithm to use the concept of events, consisting of a subject, verb, object, and other elements, in a coherent narrative. She trained the system on the storyline of such science fiction shows as Doctor Who, Futurama, and The X-Files. Then, when fed a snippet of text, it will identify events, and use them to shape a continuation of the plot churned out by a neural network. In another project, completed last year, Martin developed a way to guide a language model towards a particular event, such as two characters getting married.

Unfortunately, these systems still often get confused, and Martin doesnt think they would make a good DM. We're nowhere close to this being a reality yet, she says.

Noah Smith, a professor at the University of Washington who specializes in AI and language, says Martins work reflects a growing interest in combining two different approaches to AI: machine learning and rule-based programs. And although hes never played Dungeons & Dragons himself, Smith says creating a convincing Dungeon Master seems like a worthwhile challenge.

Sometimes grand challenge goals are helpful in getting a lot of researchers moving in a single direction, Smith says. And some of what spins out is also useful in more practical applications.

Maintaining a convincing narrative remains a fundamental and vexing problem with existing language algorithms.

Large neural networks trained to find statistical patterns in vast quantities of text scraped from the web have recently proven capable of generating convincing looking snippets of text. In February 2019, the AI company OpenAI developed a tool called GPT-2 capable of generating narratives in response to a short prompt. The output of GPT-2 could sometimes seem startlingly coherent and creative, but it also would inevitably produce weird gibberish.

More here:

Forget Chessthe Real Challenge Is Teaching AI to Play D&D - WIRED