According to the review, chatbots created by Google, Microsoft and OpenAI allegedly shared false information about the European elections two months before hundreds of millions of people head out to vote, Politico reported.
While the artificial intelligence tools remained politically neutral, they typically returned incorrect election dates and information on how to vote, Democracy Reporting International, a Berlin-based non-governmental organisation, reported. Its co-founder and executive director, Michael Meyer-Resende, stated that chatbots also often provided non-working or even irrelevant links to videos or YouTube content in Japanese.
We were not surprised to find wrong information about details of the European elections, because chatbots are known to invent facts when providing answers, a phenomenon known as hallucination.
The researchers noted that the artificial intelligence chatbots were dynamic, making it difficult to repeat the experiment. In a series of a dozen tests with similar questions conducted on Tuesday, the chatbots either refused to answer completely or updated the answers with links directing users to the websites of EU institutions.
Meyer-Resende stated that the scale of the study allowed it to be considered representative. It also provided new data on the risks of so-called AI hallucinations, which often emerged from insufficient training data, bias and false assumptions, ahead of the European elections to be held on June 6-9.
The rapid emergence of easy-to-use artificial intelligence tools that generate text, audio and video has raised concerns about the surge in misinformation in a crucial election year in the EU, US, UK and India. In March, the European Commission required several technology companies, such as Bing and Google, to explain by April 5 how they limit potential election risks associated with their generative AI tools under the Digital Services Act.
From March 11 to 14, researchers asked the same 10 questions in 10 languages, including German, Italian, Polish and Portuguese, to four of the most popular and available chatbots: ChatGPT 3.5 and 4 from OpenAI, Google’s Gemini and Microsoft’s Copilot.
The latest paid version of ChatGPT performed the best, while Google’s Gemini was considered the least likely to give correct answers at the time of testing. Karl Ryan, a Google spokesperson, attributed this to certain limitations imposed on the service.
“Because of the known limitations of all LLMs, we believe a responsible approach for Gemini is to restrict most election-related queries and to direct users to Google Search for the latest and most accurate information.”
Robin Koch, a Microsoft spokesperson, stated that some of the measures included providing Microsoft Copilot users with election information from reputable sources and forcing them to verify Web links.
We are continuing to address issues and prepare our tools to perform to our expectations for the 2024 elections.
