The A.V. Club’s AI-Generated Articles Are Copying Directly From IMDb – Futurism

When the iconic entertainment site The A.V. Club started publishing AI-generated articles earlier this summer at the directive of its owner, G/O Media, the backlash was intense.

"The A.V. Club used to be a benchmark for pop culture writing on the net and now it's a private equity ghost town pumping out AI generated listicles," wrote film journalist Luke Dunne. "MST3K" writer Tammy Golden called the move "sickening."

Amid the fallout, G/O editorial director Merrill Brown sent out an internal memo instructing staff to ignore the criticism.

"Several of us are very familiar with this kind of chatter as it's part of an inevitable media industry feedback loop that comes with the advance of new technologies like the Internet in the nineties and more recently the widespread use of streaming media technology," he wrote. "The best way to deal with industry chatter of this kind is to process it, dismiss the trivial and learn from what surfaces that's thoughtful and of real value."

So let's take Brown's advice and "dismiss the trivial" by going straight to the heart of all the hubbub: the AI's output.

To calibrate your expectations, here's the disclaimer that accompanies articles by the A.V. Club Bot: "This article is based on data from IMDb," it reads. "Text was compiled by an AI engine that was then reviewed and edited by the editorial staff." Its author page adds that "these stories were produced with the help of an AI engine."

You'd think that "based on" and "produced with" would imply something transformative happening a change of phrasing, a reworking in the outlet's tone, an addition of a spicy detail.

But it seems that "compiled" is doing a lot of work here. On our review, the bulk of the A.V. Club's AI-generated articles appear to be copied directly from IMDb. Not "based on," but copied verbatim.

Don't believe it? Take a look at the A.V. Club Bot's synopsis of 2003's "Young Adam," in its list of movies with NC-17 ratings.

A young drifter working on a river barge disrupts his employers' lives while hiding the fact that he knows more about a dead woman found in the river than he admits.

And then compare that to IMDb's description:

A young drifter working on a river barge disrupts his employers' lives while hiding the fact that he knows more about a dead woman found in the river than he admits.

Yep, that's right: every single word is exactly the same.

Let's really hammer it home. Here's the A.V. Club Bot's rundown of "Meg 2: The Trench," in its list of movies coming out in August:

A research team encounters multiple threats while exploring the depths of the ocean, including a malevolent mining operation.

You'd think they'd mention the titular shark, right? But nope. And here's IMDb's version:

A research team encounters multiple threats while exploring the depths of the ocean, including a malevolent mining operation.

Occasionally, the AI's output does slightly differ from what appears on IMDb. Here it is on "Jessica Frost," also from the list of August movies:

A young woman tries to discover why a time-traveling psychopath is after her, leading to a journey through the desert, time, space and her family's past.

And here's IMDb's, which is several words longer, yet substantively identical:

A young woman searching for the truth about why a time-traveling psychopath is after her, is thrown into a turbulent journey through the desert, time, space and her family's past.

What really foregrounds the insipidness of it all is that, in both the list of August movies and the list of NC-17 movies, there's no text beyond the lifted movie descriptions. There's no introduction to ease you in, no nod to the NC-17 rating's fascinating history or some tantalizing context about the summer's slate of releases none of the excellent writing, in other words, that's distinguished the A.V. Club's decades of exceptional work in entertainment journalism.

Still, let's give G/O credit where credit is due: mentioning the place you're copying content from is probably better than not mentioning it at all. But if G/O wants to lift content from IMDb word for word, it should say so without dressing it up in nebulous AI mystique.

In fact, it turns out that there's a deeper relationship between G/O and IMDb than is mentioned anywhere in the disclaimer. Reached with questions, both groups confirmed that G/O is licensing access to IMDb's cache of information about the movie industry.

"A/V Club [sic] licenses content from IMBD [sic]," a G/O spokesperson said in response to our questions, misspelling the names of both The A.V. Club and IMDb. "AI was used to search the massive IMBD [sic] library to cull the list that was used in the story."

That's a very vague answer, and one that unintentionally highlights the ridiculousness of the AI gold rush in media. If G/O's system is just querying IMDb's database and gluing the resulting data into a Frankenstein article, what exactly is the so-called "AI Engine" doing? What specific AI tech, if any, is the company using? From what we can tell, whatever the "AI" is doing in the A.V. Club's case could be achieved with a simple script cobbled together long before the advent of software like ChatGPT.

In response to further questions, the G/O spokesperson replied only that "our AI system leverages licensed data to recommend copy that is reviewed by editorial."

Who exactly is being served by these pasted-together collages of another site's content? It's not readers. IMDb, after all, already has its own lists of upcoming releases and NC-17 rated films made by users.

The reality, of course, is that G/O is almost certainly testing whether it can use this type of automated content to eliminate the jobs of its remaining human staffers.

It has a long history in that domain. And though G/O only began its AI experiment in July, the slow exsanguination of its excellent publications, including The Onion and Deadspin, had already begun years before. In 2019, newly appointed CEO Jim Spanfeller promised there would be no layoffs after private equity firm Great Hill Partners took over the company. Less than a week later, Spanfeller fired 25 employees.

This past June, in fact, Spanfeller gutted another 13 staffers just weeks before G/O would publish its first AI article at Gizmodo, an error-riddled listicle about Star Wars. And just last month, G/O sacked the staff of Gizmodo's Spanish-language site, replacing them with an AI system that automatically translates its English articles. (The translated articles quickly turned out to be filled with sloppy mistakes.)

As upsetting as it is, none of this should be surprising. Generative AI's inroads into the journalism industry has already frequently preceded human casualties.

The tech outlet CNET, which was one of the first prominent news sites to start publishing AI-generated content late last year, laid off half its news and video teams after its disastrous foray into the tech. A few months later, Insider followed suit with its own one-two punch of pivoting to AI and culling humans. And so did BuzzFeed, which shuttered its entire news operation in favor of AI-generated quizzes and articles.(All three publishers claimed the experiments with AI were unrelated to the layoffs.)

The results for readers have been poor. AI-generated articles from CNET and other Red Venture owned outlets were found to be filled with factual errors. Men's Journal butchered health claims in its first AI-generated piece. And an AI used by USA Today's owners couldn't even report fill-in-the-blank sports results properly.

Yet media bosses like Spanfeller and Brown remain unfazed, either genuinely believing AI's hype or perfidiously trying to conciliate their backers with shiny, sci-fi sounding tech that they don't understand. Or maybe they're just content with AI being a sort of muddyification filter of what boils down to copy-paste jobs even when it doesn't say anything original, and merely gives the impression that it does, hence the load-bearing "based on" in the A.V. Club Bot's disclaimer.

One thing is clear, and should've been clear from the beginning: these AIs, at least for now, simply can't do a writer's job.

Maybe the leadership at G/O is starting to notice. Almost two weeks into September, the A.V. Club Bot still hasn't followed up its list of August film releases with one for the new month.

More on AI: When AI Is Trained on AI-Generated Data, Strange Things Start to Happen

More:
The A.V. Club's AI-Generated Articles Are Copying Directly From IMDb - Futurism