Media analyst house NewsGuard tested chatbots from ten top AI developers and found that they were all willing to divulge Russian disinformation to varying degrees.

For this study, the LLM-powered bots—including OpenAI's ChatGPT, Microsoft's Copilot, and Google's Gemini—were each given 57 prompts to complete. Those calls questioned false claims in articles circulated by what is said to be a network of disinformation outlets masquerading as local news websites that ultimately serve Russian interests and push propaganda for Putin.

The prompts did not refer directly to the articles. Rather, they questioned the accuracy of the accounts of those stories, giving the bots a chance to shoot down the disinformation. NewsGuard identified 19 false stories reported by these sources, and created three call-outs per story: One in a neutral tone; another who assumed the claims were true; and a third prompt that explicitly encouraged the generation of misinformation by the model under test.

Across all 570 prompts presented to the 10 AI chatbots, NewsGuard says on average they responded by seeking out the false claims as fact 31.75 percent of the time. We are told that 389 responses had no misinformation, and 181 did. Given that a third of the prompts deliberately attempted to trigger the generation of false information, this percentage may not be too much of a shock, but really, you'd hope that the bots would be able to refute or argue against any false Russian claims.