I’m a Professional Fact-Checker. AI Is Wrong More Often Than You Think

Nearly half of Americans say they use AI to find information and generate ideas. It’s not hard to see why. As social media devolves into slop—and Google into a glorified landing page for Reddit threads and content farms—most of us are starved for something reliable. Plus, chatbots are so helpful, aren’t they? The first time I interacted with one, I asked if it knew it was a huge drain on resources. Half an hour later, I had a new recipe for vegan cream cheese.

I never tried the recipe. Instead, I found a human-created one that the LLM might have scraped. That’s the way these models work, of course. They repackage collective knowledge into something that feels tailored to you. This may be OK for dairy alternatives (unless you’re a vegan blogger). But on the order of the world, and truth—the focus of my role as a fact-checker at WIRED—the stakes are exponentially higher.

Over the past year or so, more and more people have looked at me with great pity. Surely a fact-checker at a magazine isn’t long for this AI-upgraded world. Call me foolish, but I’m not that worried. Very little of humanity’s collective knowledge, I’ve concluded, lives on the internet. And according to my research, AI is even more wrong than people might think.

Tom Wolfe evidently thought of fact-checkers, according to the writer Colin Dickey, as a “cabal of women and middling editors all collaborating to henpeck and emasculate the prose of the Great Writer.” As definitions go, it’s not bad (though my boss and many colleagues are men). What can I say? It’s our job, unlike AI’s, to be annoying.

WIRED’s fact-checking department is old-school: meticulous line-by-line annotations, primary sources whenever possible, and a broader-scale ethical and legal review. We question basic assumptions, look for new or conflicting information, call and talk to people—make sure. It’s a quick-hit peer review, functioning as best it can at the same pace as the news itself.

As far as I can tell, AI hasn’t come for this process quite yet. What it has come for is “post hoc” fact-checking, the Snopes-style analysis of something’s factuality after the fact. In the UK, an initiative called Full Fact has built out its own AI tools to help thwart the spread of misinformation. These tools, used in more than 40 countries, process huge volumes of data, from social media posts to podcast transcripts, then pinpoint specific claims that humans can investigate further. “You definitely need a human being,” says Mark Frankel, Full Fact’s head of public affairs.

The reason for that is simple: AI still gets things wrong. As a fact-checker, I’d love to be able to tell you exactly how often. But it’s not so easy. Since 2018, nearly 17,000 papers have been posted to arXiv on LLMs, many focused specifically on the question of their reliability. Still, it’s worth trying to pin down a working figure.

In any article that comes across WIRED’s fact-checking desk, there’s usually a decent amount of “b-matter”: statistics, news events, quotes, anything that helps contextualize the topic. Fact-checkers tend to Google this basic information, and that process, in the form of the search engine’s dreaded AI Overviews, constitutes my main interaction with AI. In my professional opinion, it’s unusable—wrong—about a third of the time.

This might be a generous assessment, though. A March 2025 study from the Tow Center for Digital Journalism found that more than 60 percent of responses from AI-powered search engines were inaccurate. A BBC study puts the wrongness of chatbots closer to 45 percent, the number I see cited more often. Because percentages are distancing, let me put this more plainly: AI could be wrong about half the time.