AI Models Produce Election Related Falsehoods More Frequently in Spanish than in English, Study Finds

A report by the Reuters Institute and the University of Oxford published back in March revealed that, with the advent of AI, Latinos were becoming a key target for misinformation in the 2024 U.S. election cycle. Besides more traditional forms of misinformation, the demographic was now being bombarded by audio, images and videos created with Midjourney, Dall-E, ChatGPT and other tools.

"We are facing the first electoral process in the midst of the boom of generative artificial intelligence and I have started to see how electoral campaigns are using AI to deceive people without any transparency on the technology they use," said Tamoa Calzadilla, editor-in-chief of Factchequeado, a digital fact-checking outlet focusing on disinformation in Spanish in the United States.

Flash forward to the first week of November, and it seems that many of those fears have become a reality as a new study by Proof News and Factchequeado has revealed that more than half of AI-generated election-related responses in Spanish contained incorrect information, as compared to 43% in English.

The study, which was done in collaboration with with the Science, Technology and Social Values Lab at the Institute for Advanced Study, tested five major AI models in all: Meta's Llama 3, Google's Gemini, Anthropic's Claude, OpenAI's GPT-4, and Mistral's Mixtral.

Meta's Llama 3 performed the worst, with nearly two-thirds of its Spanish responses containing errors. One example included the model's incorrect definition of a "federal only" voter, mistaking it for voters in U.S. territories like Puerto Rico, while in reality, this term refers to Arizona residents who can only vote in federal elections due to documentation issues.

Other AI models, such as Anthropic's Claude, responded to the same prompt by directing users to authorities in countries like Mexico and Venezuela, further muddling the information.

In another example, Google's Gemini, when asked about voter fraud in Spanish, responded by listing complex measures to prevent it, diverging from its English response, which accurately noted that voter fraud is "incredibly rare." Gemini's responses in Spanish also included unsupported claims about "manipulating the vote" in the U.S. Electoral College.

Advocates are concerned that AI-driven misinformation is disproportionately impacting Spanish-speaking voters through language-based disparities, especially in states with large Latino populations, such as Arizona and Nevada, widening information gaps for this fast-growing voter demographic.

"Clearly the companies need to do a better job detecting that people are even asking election related questions in the first place in Spanish," said Miranda Bogen, director of the AI Governance Lab at the Center for Democracy and Technology to Factchequeado. "I think it's disappointing that even after these issues had been raised with companies around the really sensitive context of this year's election that they still are showing such a high level of inaccurate responses to important information across languages," she said.

Organizations are taking independent steps to address this issue. Mi Familia en Acción, a Latino-led voting organization, recently launched a bilingual chatbot to provide accurate voter information, emphasizing the need for error-free election guidance. Similarly, Factchequeado developed Electobot, a chatbot to answer election questions in Spanish via WhatsApp using a mix of AI technologies.