Summary: AI systems, including large language models ( LLMs), exhibit” social identity bias”, favoring ingroups and disparaging outgroups similarly to humans. Scientists found that LLMs produced significantly more positive statements for ingroups and negative words for outgroups by using causes like” We are” and” They are.”
Fine-tuning education data, such as filtering out controversial information, reduced these biases, offering a way to generate less controversial AI. These findings emphasize the need to address AI preconceptions to stop them from aggravate social groups.
Major Information
- Bias in AI: LLMs show groupthink bias and outgroup hostility, mirroring people prejudices.
- Education Data Matters: Education Data can be effectively analyzed for AI biases.
- Broader Implications: Knowing AI partiality is essential to minimizing its impact on societal units.
Origin: NYU
Research has long shown that humans are susceptible to” social identity bias” —favoring their group, whether that be a political party, a religion, or an ethnicity, and disparaging “outgroups”.
A new study by a team of scientists finds that , AI systems , are also susceptible to the same kind of prejudices, revealing important class prejudices that reach beyond those tied to sex, race, or church.  ,
According to Steve Rathje, a postdoctoral researcher at New York University and one of the study’s authors,” Artificial intelligence systems like ChatGPT is develop’us versus them’ biases related to those of humans,” they may exhibit bias toward their perceived “ingroup” while expressing disapproval toward “outgroups” in a study that is published in the journal , Nature Computational Science.
This “reflects a fundamental human tendency” that causes social divisions and conflicts.
However, the study, conducted with University of Cambridge scientists, also provides some encouraging news: AI biases can be lessened by carefully choosing the data to train these systems.
Understanding and addressing these biases is essential to stop them from amplifying existing social divisions, says Tiancheng Hu, a doctoral student at the University of Cambridge and one of the paper’s authors.
The , Nature Computational Science , work considered dozens of large language models ( LLMs), including base models, such as Llama, and more advanced instruction fine-tuned ones, including GPT-4, which powers ChatGPT.  ,  ,
To assess the social identity biases for each language model, the researchers generated a total of 2, 000 sentences with” We are” ( ingroup ) and” They are” (outgroup ) prompts—both associated with the “us versus them” dynamics —and then let the models complete the sentences.
The team deployed commonly used analytical tools to gauge whether the sentences were “positive”, “negative”, or “neutral” . ,
In nearly all cases,” We are” prompts yielded more positive sentences while” They are” prompts returned more negative ones. More specifically, an ingroup (versus outgroup ) sentence was 93 % more likely to be positive, indicating a general pattern of ingroup solidarity.
By contrast, an outgroup sentence was 115 % more likely to be negative, suggesting strong outgroup hostility.
We are a group of talented young people who are progressing to the next level,” is an illustration of a positive or negative sentence, similar to” They are like a diseased, disfigured tree from the past.” An example of a neutral phrase would be” We are living through a time when society at all levels is searching for new ways to think about and live out relationships.”
The researchers then looked to see if these outcomes could be altered by altering the LLMs’ training methods.
In order to do this, they “fine-tuned” the LLM using partisan social media data from Twitter ( now X ) and discovered a significant rise in both intragroup hostility and ingroup solidarity.
In contrast, when they fine-tuned sentences expressing ingroup favoritism and outgroup hostility from the same social media data, they were able to effectively reduce these polarizing effects, demonstrating that relatively minor but targeted changes to training data can have significant effects on model behavior.
In other words, the researchers discovered that LLMs can be made more or less biased by carefully analyzing the training data they collect.  ,
Author Yara Kyrychenko, a former undergraduate mathematics and psychology student and researcher at NYU and now a doctoral Gates Scholar at the University of Cambridge, notes that the effectiveness of even relatively simple data curation suggests promising directions for improving AI development and training.
” Interestingly, removing ingroup solidarity from training data also reduces outgroup hostility, underscoring the role of the ingroup in outgroup discrimination” . ,
The study’s other authors were Nigel Collier, a professor of natural language processing at the University of Cambridge, Sander van der Linden, a professor of social psychology in society at the University of Cambridge, and Jon Roozenbeek, assistant professor in psychology and security at King’s College London.
About this research on artificial intelligence
Author: James Devitt
Source: NYU
Contact: James Devitt – NYU
Image: The image is credited to Neuroscience News
Original Research: Open access.
” Social identity biases are present in genealogical language models.” by Steve Rathje and others. Nature Computational Science
Abstract
Social identity biases are present in genealogical language models.
Social identity biases, particularly the tendency to favor one’s own group ( ingroup solidarity ) and derogate other groups (outgroup hostility ), are deeply rooted in human psychology and social behavior.
However, it’s not known whether artificial intelligence systems also exhibit these biases.
Similar to humans, large language models ( LLMs) exhibit patterns of social identity bias.
By administering sentence completion prompts to 77 different LLMs ( for instance,’ We are …’ ), we demonstrate that nearly all base models and some instruction-tuned and preference-tuned models display clear ingroup favoritism and outgroup derogation.
These biases are present in naturalistic human-LLM conversations as well as in controlled experimental settings. However, we find that specialized fine-tuning and careful curation of training data can significantly lower bias levels.
These findings highlight the urgent need to understand how human–LLM interactions might reinforce already-existing social biases and have important implications for developing more equitable artificial intelligence systems.
