THERE ARE MANY photos of Tom Hanks, but none like the images of the leading everyman shown at the Black Hat computer security conference Wednesday: They were made by machine-learning algorithms, not a camera.
Philip Tully, a data scientist at security company FireEye, generated the hoax Hankses to test how easily open-source software from artificial intelligence labs could be adapted to misinformation campaigns. His conclusion: “People with not a lot of experience can take these machine-learning models and do pretty powerful things with them,” he says.
Seen at full resolution, FireEye’s fake Hanks images have flaws like unnatural neck folds and skin textures. But they accurately reproduce the familiar details of the actor’s face like his brow furrows and green-gray eyes, which gaze cooly at the viewer. At the scale of a social network thumbnail, the AI-made images could easily pass as real.
To make them, Tully needed only to gather a few hundred images of Hanks online and spend less than $100 to tune open-source face-generation software to his chosen subject. Armed with the tweaked software, he cranks out Hanks. Tully also used other open-source AI software to attempt to mimic the actor’s voice from three YouTube clips, with less impressive results.
By demonstrating just how cheaply and easily a person can generate passable fake photos, the FireEye project could add weight to concerns that online disinformation could be magnified by AI technology that generates passable images or speech. Those techniques and their output are often called deepfakes, a term taken from the name of a Reddit account that late in 2017 posted pornographic videos modified to include the faces of Hollywood actresses.
Most deepfakes observed in the wilds of the internet are low quality and created for pornographic or entertainment purposes. So far, the best-documented malicious use of deepfakes is harassment of women. Corporate projects or media productions can create slicker output, including videos, on bigger budgets. FireEye’s researchers wanted to show how someone could piggyback on sophisticated AI research with minimal resources or AI expertise. Members of Congress from both parties have raised concerns that deepfakes could be bent for political interference.
Tully’s deepfake experiments took advantage of the way academic and corporate AI research groups openly publish their latest advances and often release their code. He used a technique known as fine-tuning in which a machine-learning model built at great expense with a large data set of examples is adapted to a specific task with a much smaller pool of examples.
To make the fake Hanks, Tully adapted a face-generation model released by Nvidia last year. The chip company made its software by processing millions of example faces over several days on a cluster of powerful graphics processors. Tully adapted it into a Hanks-generator in less than a day on a single graphics processor rented in the cloud. Separately, he cloned Hanks’ voice in minutes using only his laptop, three 30-second audio clips, and a grad student's open-source recreation of a Google voice-synthesis project.
A "deepfake" audio clip of Tom Hanks created by FireEye.
As competition among AI labs drives further advances—and those results are shared—such projects will get more and more convincing, he says. “If this continues, there could be negative consequences for society at large,” Tully says. He previously worked with an intern to show that AI text-generation software could create content similar to that produced by Russia’s Internet Research Agency, which attempted to manipulate the 2016 presidential election.
Tim Hwang, a research fellow at the Center for Security and Emerging Technology at Georgetown, says experiments like FireEye’s can help anchor the debate over the threat of deepfakes, which sometimes borders on hysteria. “A lot of discussion about the threat has been driven by dramatic anecdotes,” he says. A recurring example is of an imagined presidential election being rigged by a flawless, well-timed deepfake of one of the candidates.
FireEye’s project fills in some practical detail by showing what can be achieved with minimal resources and the open-source fruits of AI research. Hwang says that combining that kind of information with what’s known about how disinformation organizations operate is a better way to assess the deepfake threat than considering worst-case scenarios. “I think about who’s the middle manager at the Internet Research Agency who has to present to the higher ups and say ‘Here’s what you got for your investment,’” Hwang says.
“These don’t have to be perfect for them to be convincing in a world where we rapidly consume information in the way we do.”
LEE FOSTER, FIREEYE
Despite Tully’s faux Hankses, Hwang says it looks like the killer app for deepfake disinformation is yet to arrive. The IRA and others accomplish a lot with cheap labor and relatively simple tech infrastructure and probably don’t have much to gain from even small AI projects. Creating sophisticated deepfake video would require significant time and expertise. Hwang authored a report published last month that concludes deepfakes don’t present an acute and imminent threat, but that society should invest in defenses anyway.
The report recommends corporate and academic labs create “deepfake zoos” that collect examples made with different open-source techniques like those used by FireEye to help create deepfake detectors. Some companies have already started work on similar projects. Nvidia has published results on how to detect faces synthesized with its software.
Facebook recently created a trove of deepfake video and offered $500,000 for the best performing deepfake detector trained on them. The winner could spot deepfakes not in Facebook’s collection only 65 percent of the time.
FireEye’s researchers are also thinking about how deepfakes might fit in with existing practices in the disinformation business. Tully’s talk on Wednesday is a coproduction with Lee Foster, who leads a team at the company that tracks online manipulation campaigns. Last year, it unmasked a sprawling pro-Iran disinformation campaign that used fake personas on Twitter and Facebook and even tricked experts on Middle East politics into giving interviews later used to push the operation’s message.
Foster says Tully’s results and his own experience with disinformation sowers makes him think they could soon turn to deepfakes. Profile photos that have been lifted from other sources are a clue that disinformation investigators use to help unmask fake personas. The fake Hanks content is not far from the quality needed to provide tricksters an alternative, he says. “These don’t have to be perfect for them to be convincing in a world where we rapidly consume information in the way we do,” Foster says. “If you’re scrolling through your Twitter feed at speed, you’re not going to be closely scrutinizing the profile pictures.”
No comments:
Post a Comment