No LLM Can Be Neutral: AI Chatbots and Elections in Brazil and Beyond

Leia em português aqui.

A few days ago, I was surprised to learn from a podcast that Brazil is OpenAI’s third-largest market in the world. Brazilian users send more than 215 million queries a day to ChatGPT, which shows how widespread the use of this technology is in the country. The podcast in question is an episode of the series Café da Manhã (A Máquina), produced by Folha. In it, journalist Patrícia Campos Mello interviews Bruno Lewicki, OpenAI’s director of public policy for Latin America. As a researcher in computational linguistics with Brazilian origins who has spent a few years investigating the political opinions present in language models such as ChatGPT, I felt the urge to offer a critical reading of this episode.

The interview contains many passages in which both the interviewer and the interviewee normalize the use of AI in elections or show great enthusiasm for it. So much optimism ignores (or rather, omits) the risks that using these models during elections, or for political matters in general, poses to users. Statements uttered by Bruno Lewicki such as “It [the use of these technologies during the elections] can and should be regarded as perfectly natural,” “we approach this process very calmly,” or “It’s a source of pride for those of us who work at OpenAI in Latin America, Brazil is becoming a great elections case study” distort the reality of this kind of technology and stifle the questioning we should be doing far more often about its use in our daily lives. In fact, instead of questioning the real risks these models pose to the information ecosystem, Patrícia asks Bruno for his opinion on the case of the Anthropic employee who resigned out of fear that humanity will be extinct by the end of the decade. This is yet another way of diverting attention from the problematic issues we are facing right now, such as content generated with false information, the sensitive user data these companies accumulate, and potential manipulation of information.

Another issue raised in the podcast is the rules imposed by the Superior Electoral Court (TSE) for the 2026 elections. According to the TSE, language models may not rank, recommend, favor, or disfavor candidates. This is a resolution that has no case law yet, meaning there are no concrete indications of how it should be implemented, and it is left to the companies to decide how to comply with it. Regarding ChatGPT’s performance on this front, Bruno Lewicki mentioned the results of a study that OpenAI commissioned from a renowned professor of electoral law. With all due respect to my fellow researcher, the evaluation of language models is now a science in its own right and requires a specialist in the field to conduct research with robust analysis. Moreover, the study mentioned has not been published, which means it has not been peer-reviewed. All of this raises many doubts about what the results of this study actually represent.

The podcast also contains inaccurate statements by Bruno Lewicki about how these models are trained. During the interview, he says: “The model is already trained to be unbiased, especially when it comes to political and electoral material.” The reality is that the model is not trained to be unbiased, because that is, in fact, impossible. The model is corrected after the training stages to exhibit different biases (opinions) or to answer specific questions in specific ways. That does not mean it is “unbiased.”

First, let’s start from the assumption that being unbiased means holding a neutral point of view. What does it mean to hold a neutral point of view regarding political issues? It certainly doesn’t mean to be center because this is per se a political position. So perhaps being neutral would be to be objective, but what does objectivity mean? Borrowing the provocation from film director Jean-Luc Godard, “Objectivity is five minutes for Hitler, five minutes for the Jews”. This means giving the same space all opinions. However, there are opinions that are not valid, that go against constitutions and human rights. Therefore, neutrality in this sense is also not possible. Let’s then say that neutrality is giving space to valid arguments for and against an issue (arguments that do not violate a given constitution or human rights). Yet, even this does not achieve neutrality, because text generated in support of or against something is a selection of opinions. Every selection, by its very nature, involves choices and is therefore not neutral. In the case of LLMs, if opinions aren’t deliberately chosen, the model will reflect whatever appears most frequently in the training data.

Second, from a technical standpoint, these models go through three training stages, and in none of them does the model deliberately “learn” to be neutral. The first is pre-training, in which the model learns numerical representations of natural language by predicting the next word. This process uses enormous amounts of text collected from the internet, ranging from blog posts (which account for most of the files) and forum comments to scientific articles (which account for a very small proportion). Next comes the first post-training stage, in which the model learns to chat by predicting the next word in data formatted as dialogues. Finally, there is a last post-training stage, usually involving reinforcement learning, in which the model learns human preferences and is optimized to be “helpful, harmless, and honest”. However, the last phase in frontier labs involves a lot more than optimizing models to be “helpful, harmless, and honest”.

Elon Musk, for example, has publicly complained on X about how Grok, his company’s model, answers certain questions. When Musk sees a Grok answer he doesn’t like and writes on X “will fix it tomorrow,” what he is really saying is: “I’m going to ask the post-training team to further train the model to answer this specific question the way I find it appropriate”. The team cannot, for example, retrain the model from scratch, because that would require a tremendous amount of computing resources and because it would mean selectively removing a great deal of data from the training corpus, which is also no easy task, since no one knows exactly where the model learned a given piece of information. In short, it is relatively easy to change the model’s opinion on extremely specific questions, but it is hard to influence many opinions across the board. This is for two reasons: because the questions users can ask are infinite, and most importantly, because building high-quality datasets that address specific opinions on a wide range of subjects while maintaining the model’s performance in terms of factuality is, arguably, impossible.

I conclude my critical reading of the podcast with a word of caution to users: it is essential to keep a critical eye on the information received during interactions with these models and to seek out reliable, recognized primary sources, as I have written before. However, this responsibility cannot fall on users alone. That is why I will only be optimistic about these technologies when they are finally regulated by laws requiring them to be fully open. Yes, I really mean fully open. That is, if companies want to operate, they have to document and make publicly available all the data and code used in their models’ training regime. Transparency is the key to understanding the behavior of these models, and as of now, the only way to be sure that information is not being intentionally manipulated by companies or authoritarian governments in favor of their own interests.