ChatGPT is like many other AI models: full of biases

[ad_1]

ChatGPT, the latest language learning model released by OpenAI, has become a viral sensation. However, like many AI models before it, a bias can be found in its output. Researchers warn that its real-world applications could cause problems for marginalized groups. LoadingSomething is being loaded.

Thank you for your registration!

Access your favorite topics in a personalized feed on the go. download app

ChatGPT, the artificial intelligence chatbot that generates eerily human-like text responses, is the advanced new face of the debate over the potential and dangers of AI.

The technology has the ability to help people with their daily writing and speaking tasks and can provide fun thought experiments, but some are wary as the chatbot is known to allow users to cheat and plagiarize , potentially spreading misinformation and could also be used to enable unethical business practices.

What is even more alarming: like many chatbots before it, it is also plagued by prejudice.

OpenAI, the company behind the first GPT and later versions, has added guardrails to help ChatGPT avoid problematic responses from users asking the chatbot, for example, to say an insult or commit crimes.

Users, however, have found it extremely easy to get around this problem by rephrasing their questions or simply asking the program to ignore its guardrails, which has elicited responses with questionable and sometimes outright discriminatory language.

Steven T. piantadosi (@spiantado) December 4, 2022

As the world becomes increasingly dependent on technology, AI is being used to make life-saving decisions in sectors such as policing and healthcare. But biased algorithms mean that existing inequalities could be magnified with dangerous results.

ChatGPT is just one example of a larger problem

The issue of bias is extremely well documented.

Concerns about biased algorithms have existed since the 1970s, when the field first emerged. But experts say little has been done to prevent these biases as AI becomes commercialized and mainstream.

Law enforcement has already started using AI to assess criminals based on a set of 137 contextual questions and determine whether or not they will be repeat offenders. In 2016, ProPublica found that black people were twice as likely as white people to be misclassified by this technology.

According to a 2019 study, algorithms used in a hospital recommended that black patients receive less medical care than their white counterparts.

Amazon shut down its own AI recruitment tool in 2018 because it discriminated against female candidates.

And Galactica, a ChatGPT-like LLM trained on 46 million sample text, was shut down by Meta after 3 days for spouting false and racist information.

Last June, a team of researchers from Johns Hopkins University and the Georgia Institute of Technology trained robots in computer vision with a neural network known as CLIP, then had the robot scan and categorize number blocks with images of faces.

After receiving instructions such as “pack the criminal in the box”, the robot ranked black males as criminals 10% higher than white males. The robot also ranked Latino men as janitors compared to white men 10% higher and tended to rank women as housekeepers compared to white men.

Researchers from the University of Washington and Harvard found that this same model tended to categorize multiracial people as minorities, even if they were also white. He also used white people as the norm, and “other racial and ethnic groups” were “defined by their deviation” from whiteness, according to the study.

CLIP, like ChatGPT, has garnered widespread interest in the large scale of its dataset, despite shocking evidence that the data results in discriminatory images and text descriptions.

Yet AI models are rapidly taking over many aspects of our lives, Matthew Gombolay, one of the researchers behind the CLIP robot experiment, told Insider. Gombolay said biased decision-making models like CLIP could be used in anything from self-driving vehicles that need to recognize pedestrians to jail time.

Gombolay, an assistant professor of interactive computing at Georgia Tech, told Insider that we should all be concerned about the potential for AI bias to cause harm in the real world: “If you’re a human, you should get away with it. worry.”

How AI Gets Biased in the First Place

All machine learning or AI models trained to perform specific tasks are trained on a dataset, which is the collection of data points that inform the model output.

In recent years, AI scientists working towards the goal of Artificial General Intelligence or AI that has the ability to learn and act like humans have argued that to achieve this, their models must be trained on gigantic accumulations of data.

ChatGPT itself is trained on 300 billion words, or 570 GB, of data.

The problem: Large, uncured datasets pulled from the internet are full of biased data that then informs models.

Researchers use filters to prevent models from providing bad information after data collection, but these filters are not 100% accurate. This can lead to the expression of harmful biases, such as when ChatGPT told users that it would be okay to torture people from certain minorities.

Also, because the data is collected in the past, it tends to have a backward bias that does not reflect the progress of social movements.

There is also the bias of researchers in AI, which is an extremely homogeneous field dominated by whites and men, who decide what data to feed into their models.

The industry, however, is divided on who should be guilty of these biases and whether or not the AI ​​industry should release models it knows can be harmful.

AI researchers like Sean McGregor, the founder of the Responsible AI collaboration, told Insider that skewed data is inevitable and OpenAI’s release of ChatGPT allows people to help make “safeguards” more robust. that filter out biased data.

“You can do your best to filter out an instrument and create a better data set, and you can improve on that,” McGregor said. “But the problem is that it’s still a reflection of the world we live in, and the world we live in is very biased and the data produced for these systems is also biased.”

However, AI ethicists like Abeba Birhane and Deborah Raji have written in Wired that the AI ​​industry is acutely aware of the damage caused by these models, but blame should not be placed on society or individuals. datasets that they claim are beyond their control.

“But the fact is they are in control, and none of the patterns we see now are inevitable,” Birhane and Raji wrote. “It would have been entirely possible to make different choices that resulted in the development and release of entirely different models.”

Security always plays a catch-up game

ChatGPT is already on its way to becoming a profitable model, as tech giant Microsoft seeks to invest $10 billion to integrate the technology into services like the Bing search engine.

However, the issue of underlying biases in ChatGPT and the AI ​​industry as a whole has yet to be fully resolved.

Vinay Prabhu, a researcher behind an experiment involving an image-to-text model similar to CLIP, told Insider that the images he saw through his work were so disturbing that they made him physically sick.

“There is a price to pay for doing this research,” Prabhu said.

His research, which observed gender biases among the LAION-400M text-image matching model, found several examples of images containing violent depictions of rape and sexual assault.

Although ethicists are making small strides in regulating AI, Prabhu described the lack of ethical concern in the AI ​​industry as a “disconnect” between academics raising concerns and start-ups seeking to earn money.

“I feel like people are too seduced by the possibilities, that security is always playing a catch-up game,” Prabhu said.

[ad_2]