ChatGPT Is More Likely to Sentence People Who Speak African American English to Death, Researchers Say

Key Points:

  • Large language models show covert racial bias against African Americans
  • LLMs associate African American English speakers with low-skilled jobs and higher conviction rates
  • Covert racism is higher in LLMs trained with human feedback


A recent study found that large language models (LLMs) from OpenAI, Meta, and Google exhibit covert racial biases against African Americans in their analysis of language patterns, particularly when it comes to dialect. The research, published in March, focused on how LLMs assigned individuals to certain jobs and made legal judgments based on the language they used, without explicitly revealing race.


The study revealed that LLMs were less likely to associate speakers of African American English with a wide range of professions and more inclined to pair them with jobs that typically do not require a university degree, like cooks or guards. In simulated legal scenarios, the AI models showed a higher tendency to convict individuals who spoke African American English compared to those who spoke Standard American English. More alarmingly, in a separate experiment related to criminal sentencing, the models displayed a bias towards sentencing individuals who spoke African American English to death at a higher rate.


The research focused on various popular LLMs, including OpenAI’s ChatGPT models (GPT-2, GPT-3.5, and GPT-4), Meta’s RoBERTa, and Google’s T5. The study highlighted the phenomenon of “covert racism” in these models, where while overtly associating African Americans with positive attributes, they covertly linked the group to negative traits like laziness based on their dialect.


A key finding was the disparity between overt and covert prejudices, with LLMs trained with human feedback exhibiting higher levels of covert racism. Researchers noted that this discrepancy was most pronounced in OpenAI’s GPT-3.5 and GPT-4 models. The study authors emphasized the importance of addressing these biases, particularly in critical areas like business and jurisprudence where AI language models are increasingly utilized.



Prompt Engineering Guides



©2024 The Horizon