Self-driving car algorithms may have a problem with racism

[ad_1]

Tesla recently announced the latest version of its self-driving car software, following the software’s role in a dozen reported crashes with emergency vehicles that are under investigation by the federal agency. While these collisions have occurred for a variety of reasons, a major factor is that the artificial intelligence driving the car is not used to seeing flashing lights and vehicles being pulled over the shoulder, so algorithms under- underlying reactions are unpredictable and catastrophic.

Modern AI systems are trained on massive datasets of photographs and video footage from various sources, and use this training to determine appropriate behavior. But, if the images don’t include many examples of specific behaviors, like how to slow down near emergency vehicles, the AI ​​won’t learn the appropriate behaviors. Thus, they crash into ambulances.

Given these types of disastrous failures, a recent trend in machine learning is to identify these overlooked cases and create synthetic training data to help AI learn. Using the same algorithms that Hollywood used to put the Incredible Hulk together in The Avengers: Endgame from a stream of ones and zeros, photorealistic footage of emergency vehicles that never existed in the real world. lives are evoked from the digital ether and transmitted to AI.

I’ve designed and used these algorithms for the past 20 years, starting with the software used to generate the sorting hat in Harry Potter and the Sorcerer’s Stone, to the recent Pixar movies where I was. principal investigator.

Using these algorithms to train AIs is extremely dangerous, as they were specifically designed to represent white humans. All the sophisticated physics, computing, and statistics behind this software have been designed to realistically represent the diffuse glow of pale, white skin and the sleek highlights of long, straight hair. In contrast, researchers in computer graphics have not systematically studied the shine and shine that characterize dark and black skin, nor the characteristics of hair with an afro texture. As a result, the physics of these visual phenomena is not encoded in Hollywood algorithms.

Certainly, synthetic blacks have been portrayed in movies, like last year’s Pixar Soul movie. But behind the scenes, lighting designers found they had to push the software way beyond its default settings and learn all the new lighting techniques to create these characters. These tools weren’t designed to make humans non-white; even the most technically sophisticated artists in the world have strived to use them effectively.

Regardless, these same white human generation algorithms are currently being used by start-ups like Datagen and Synthesis AI to generate various human data sets specifically for consumption by the AIs of tomorrow. A critical examination of some of their results reveals the same trends. White skin is faithfully depicted, but the characteristic glow of black skin is either worryingly missing or painfully overlit.

Once the data from these flawed algorithms is ingested by AIs, the provenance of their malfunctions will become almost impossible to diagnose. When the Tesla Roadsters start to run disproportionately on black paramedics or Oakland residents with natural hairstyles, the cars won’t be able to signal that no one has told me what black skin looks like in real life. The behavior of artificial neural networks is notoriously difficult to trace to specific problems in their learning sets, making the source of the problem extremely opaque.

Synthetic training data is a handy shortcut when real-world collection is too expensive. But AI practitioners should ask themselves: Given the possible consequences, is it worth it? If the answer is no, they should be pushing to do it the hard way: by collecting real-world data.

Hollywood should do its part and invest in the research and development of algorithms that are rigorously, measurable, and demonstrably capable of representing all of humanity. Not only will this expand the range of stories that can be told, it could literally save someone’s life. Otherwise, even if you admit that black lives matter, your car won’t do it very quickly.

Theodore Kim is Associate Professor of Computer Science at Yale University. @TheodoreKim

[ad_2]