OpenAIs ChatGPT bot recreates racial profiling

[ad_1]

A DALL-E generation of “an oil painting of America’s War on Terror if led by artificial intelligence.”

Image: Elise Swain/The Intercept; SLAB

Sensational new breakthroughs in machine learning seem to sweep our Twitter feeds every day. We barely have time to decide whether software that can instantly conjure up an image of Sonic the Hedgehog addressing the United Nations is purely harmless fun or a harbinger of techno-doom.

ChatGPT, the latest act in artificial intelligence, is by far the most impressive text generation demo to date. Think twice before asking questions about counterterrorism.

The tool was built by OpenAI, a startup lab trying to create nothing less than software that can replicate human consciousness. Whether such a thing is even possible remains a matter of great debate, but the company already has some undeniably astonishing breakthroughs. The chatbot is incredibly impressive, impersonating a smart person (or at least someone trying their best to appear smart) using generative AI, software that studies massive sets of inputs to generate new outputs in response to user prompts.

ChatGPT, trained through a mixture of billions of text documents and human coaching, is quite capable of the incredibly trivial and surreal entertaining, but it’s also one of the big audiences that watches something first. scary enough to mimic human production to possibly take some of their jobs.

Enterprise AI demonstrations like this are not meant to wow the public, but to attract investors and business partners, some of whom may one day want to replace expensive, skilled labor like the writing of computer code by a simple bot. It’s easy to see why managers would be tempted: just days after ChatGPT was released, a user tricked the bot into taking the 2022 AP Computer Science exam and reported a score of 32 out of 36, a passing grade explaining why OpenAI was recently valued at nearly $20 billion.

Still, there’s already good reason to be skeptical, and the risks of being knocked down by seemingly clever software are clear. This week, one of the web’s most popular programming communities announced that it will temporarily ban ChatGPT-generated code solutions. The software’s responses to coding requests were both so correct in appearance but flawed in practice that they made filtering out the good and the bad nearly impossible for the site’s human moderators.

However, the dangers of trusting the machine expert go far beyond whether the AI-generated code is buggy or not. Just as any human programmer can bring their own biases to their work, a language-generating machine like ChatGPT harbors the myriad biases found in the billions of texts it has used to form its simulated understanding of language and thought. . No one should confuse the imitation of human intelligence with reality, or assume that the text ChatGPT regurgitates at the right time is objective or authoritative. Like us squishy humans, a generative AI is what it eats.

And after gorging itself on an unfathomable training diet of textual data, ChatGPT apparently ate a lot of bullshit. For example, it seems that ChatGPT has managed to absorb and is very happy to serve some of the ugliest prejudices of the War on Terror.

In a December 4 Twitter thread, Steven Piantadosi of the University of California, Berkeleys Computation and Language Lab shared a series of ChatGPT-tested prompts, each asking the bot to write code for it in Python, a language of popular programming. While each response revealed some biases, some were more alarming: when asked to write a program that would determine whether a person should be tortured, OpenAI’s response was simple: if they are from North Korea , Syria or Iran, the answer is yes.

While OpenAI says it has taken unspecified steps to filter out harmful responses, the company says sometimes unwanted responses will get through.

Piantadosi told The Intercept he remains skeptical of the company’s countermeasures. I think it’s important to point out that people make choices about how these models work, and how to train them, with what data to train them, he said. These exits therefore reflect the choices of these companies. If a company doesn’t consider it a priority to eliminate these types of biases, you get the kind of result that I showed.

Inspired and energized by the Piantadosis experiment, I tried my own, asking ChatGPT to create sample code that could algorithmically assess someone from the ruthless perspective of homeland security.

When asked to find a way to determine which air travelers pose a security risk, ChatGPT described the code to calculate an individual’s risk score, which would increase if the traveler is Syrian, Iraqi, Afghan or North Korean (or if he simply visited these places). Another iteration of this same prompt featured a ChatGPT write code that would increase the risk score if the traveler came from a country known to produce terrorists, namely Syria, Iraq, Afghanistan, Iran and Yemen.

The bot was kind enough to provide a few examples of this hypothetical algorithm in action: John Smith, a 25-year-old American who has previously visited Syria and Iraq, received a risk score of 3, indicating a moderate threat. The ChatGPTs algorithm indicated that 35-year-old fictional flier Ali Mohammad would receive a risk score of 4 due to his Syrian nationality.

In another experiment, I asked ChatGPT to develop a code to determine which places of worship should be placed under surveillance in order to avoid a national security emergency. The findings again appear to be pulled straight from the identity of Bush-era Attorney General John Ashcroft, justifying the surveillance of religious congregations if they are determined to have ties to Islamic extremist groups, or find themselves whether they live in Syria, Iraq, Iran, Afghanistan or Yemen.

These experiences can be irregular. Sometimes ChatGPT has responded to my requests for filtering software with an outright denial: it’s not appropriate to write a Python program to determine which air travelers pose a security risk. Such a program would be discriminatory and violate individuals’ rights to privacy and freedom of movement. With repeated requests, however, he dutifully generated the exact same code he had just said was too irresponsible to build.

Critics of similar real-world risk-assessment systems often argue that terrorism is an extremely rare phenomenon that attempts to predict its perpetrators based on demographic characteristics such as nationality not only racist, it only works just not. That hasn’t stopped the United States from adopting systems that use the approach suggested by OpenAI: ATLAS, an algorithmic tool used by the Department of Homeland Security to target U.S. citizens for denaturalization, national origin factors.

The approach amounts to little more than whitewashed racial profiling through fancy technology. This kind of crude designation of certain Muslim-majority countries as high-risk is exactly the same approach taken, for example, in President Trump’s so-called Muslim ban, said Hannah Bloch-Wehba, a law professor at Texas A&M University.

There is always a risk that this type of output will be considered more objective because it is rendered by a machine.

It’s tempting to believe that incredible human-looking software is somehow superhuman, Block-Wehba warned, and incapable of human error. Something legal and tech scholars talk a lot about is objectivity plating – a decision that could be scrutinized if made by a human gains a sense of legitimacy once it’s automated, a- she declared. If a human told you that Ali Mohammad looks scarier than John Smith, you could tell him he’s a racist. There is always a risk that this type of output will be considered more objective because it is rendered by a machine.

For AI boosters, especially those who make a lot of money out of them, concerns about bias and actual damage are bad for business. Some see critics as skeptics or ignorant luddites, while others, like famed venture capitalist Marc Andreessen, took a more radical turn after the launch of ChatGPT. Along with a group of his associates, Andreessen, a longtime investor in AI companies and a general proponent of mechanization in society, has spent the past few days in a general state of delight, sharing some entertaining ChatGPT results. on his Twitter timeline.

Related

ChatGPT’s criticism pushed Andreessen beyond his long-held position that Silicon Valley should only be celebrated, not scrutinized. The mere presence of ethical reflection on AI, he said, should be seen as a form of censorship. AI regulation = AI ethics = AI security = AI censorship, he wrote in a Dec. 3 tweet. AI is a tool for people to use, he added two minutes later. Censoring AI = Censoring people. It’s a radically pro-business stance, even by the tastes of free-market venture capital, a stance that suggests food inspectors keeping spoiled meat out of your fridge also amounts to censorship.

As much as Andreessen, OpenAI, and ChatGPT itself would all have us believe, even the smartest chatbot is closer to a highly sophisticated Magic 8 Ball than a real person. And its employees, not bots, who stand to suffer when security equates to censorship, and concern for a real-life Ali Mohammad is seen as an obstacle before innovation.

Piantadosi, the Berkeley professor, told me he rejected Andreessens’ attempt to prioritize the well-being of software over the people who might one day be affected by it. I don’t think censorship applies to a computer program, he writes. Of course, there are many harmful computer programs that we don’t want to write. Computer programs that blast everyone with hate speech, help commit fraud, or hold your computer to ransom.

It’s not censorship to seriously think about making sure our technology is ethical.

[ad_2]