Crawford recently discussed these issues with WIRED senior writer Tom Simonite. An edited transcript follows.
WIRED: Few people understand all the technical details of artificial intelligence. You argue that some experts working on the technology misunderstand AI more deeply.
KATE CRAWFORD: It is presented as this ethereal and objective way of making decisions, something that we can plug into everything from teaching kids to deciding who gets bail. But the name is deceptive: AI is neither artificial nor intelligent.
AI is made from vast amounts of natural resources, fuel, and human labor. And it's not intelligent in any kind of human intelligence way. It’s not able to discern things without extensive human training, and it has a completely different statistical logic for how meaning is made. Since the very beginning of AI back in 1956, we’ve made this terrible error, a sort of original sin of the field, to believe that minds are like computers and vice versa. We assume these things are an analog to human intelligence, and nothing could be further from the truth.
You take on that myth by showing how AI is constructed. Like many industrial processes it turns out to be messy. Some machine learning systems are built with hastily collected data, which can cause problems like face recognition services more error prone on minorities.
We need to look at the nose to tail production of artificial intelligence. The seeds of the data problem were planted in the 1980s, when it became common to use data sets without close knowledge of what was inside, or concern for privacy. It was just “raw” material, reused across thousands of projects.
This evolved into an ideology of mass data extraction, but data isn’t an inert substance—it always brings a context and a politics. Sentences from Reddit will be different from those in kids’ books. Images from mugshot databases have different histories than those from the Oscars, but they are all used alike. This causes a host of problems downstream. In 2021, there's still no industry-wide standard to note what kinds of data are held in training sets, how it was acquired, or potential ethical issues.
You trace the roots of emotion recognition software to dubious science funded by the Department of Defense in the 1960s. A recent review of more than 1,000 research papers found no evidence a person’s emotions can be reliably inferred from their face.
Emotion detection represents the fantasy that technology will finally answer questions that we have about human nature that are not technical questions at all. This idea that’s so contested in the field of psychology made the jump into machine learning because it is a simple theory that fits the tools. Recording people's faces and correlating that to simple, predefined, emotional states works with machine learning—if you drop culture and context and that you might change the way you look and feel hundreds of times a day.
That also becomes a feedback loop: Because we have emotion detection tools, people say we want to apply it in schools and courtrooms and to catch potential shoplifters. Recently companies are using the pandemic as a pretext to use emotion recognition on kids in schools. This takes us back to the phrenological past, this belief that you detect character and personality from the face and the skull shape.
I can’t speak to what happened inside Google, but what I’ve seen is incredibly troubling. It’s important that we have researchers inside technology companies seeing how these systems work and publishing about it.
We’ve seen research focused too narrowly on technical fixes and narrow mathematical approaches to bias, rather than a wider-lensed view of how these systems integrate with complex and high stakes social institutions like criminal justice, education, and health care. I would love to see research focus less on questions of ethics and more on questions of power. These systems are being used by powerful interests who already represent the most privileged in the world.
Is AI still useful?
Let's be clear: Statistical prediction is incredibly useful; so is an Excel spreadsheet. But it comes with its own logic, its own politics, its own ideologies that people are rarely made aware of.
And we’re relying on systems that don’t have the sort of safety rails you would expect for something so influential in everyday life. We have a regulation emergency: There are tools actually causing harm that are completely unregulated.
Do you see that changing soon?
We’re getting closer. We have Alondra Nelson in the White House Office of Science and Technology Policy, who has written about the fact that you cannot escape the politics of technology. And we’re starting to see a new coalition of activists and researchers seeing that the interrelatedness of capitalism and computation is core to climate justice, and labor rights, and racial justice. I’m optimistic.
No comments:
Post a Comment