Alec Ross
The pentagon is not the most inviting place for first-time visitors, and it was no different for Chris Lynch. When he rode the escalator out of the Pentagon metro station, Lynch was greeted by guard dogs and security personnel wearing body armor and toting machine guns. He lost cell service upon entering the building and was forced to run through more than a half mile of hallways to make his meeting in the office of the secretary of defense. He showed up late and out of breath, his hoodie and gym shoes soaked with sweat.
It was a surreal experience, Lynch told me, and it marked the beginning of “the most delightful detour of my entire life.”
Lynch had just completed a 45-day posting in the United States Digital Service, an organization formed in 2014 to fill what many officials viewed as a crucial gap in the government’s technology expertise. That year, the White House had launched HealthCare.gov to help enroll Americans in government health insurance, but it had been a technological debacle that almost derailed the Affordable Care Act. The website was so buggy that on its first day, only six people were able to sign up through the site. In response, and to prevent similar flops from occurring in the future, the White House created the USDS. The group is meant to act as a SWAT team of technologists who can come in whenever a government system needs fixing.
Lynch’s first project at the USDS involved building software to let the Pentagon and the Veterans Affairs Department more reliably share veterans’ medical records. The problem his team sought to solve was simple but had severe consequences—the VA could accept the records only in PDF format, but sometimes the Pentagon would send them as JPEGs. As a result, doctors sometimes mistreated patients and overlooked underlying conditions merely because they had incomplete records. “If you have cancer,” Lynch said, “it could be literally the difference between life or death.”
Lynch and his team set about building file-conversion software that would reformat such misfiled records, and the effort was a success—so much so that when Defense Secretary Ash Carter wanted to spin up his own military-focused branch of the USDS, the Defense Digital Service, he tapped Lynch to lead it.
The Defense Digital Service was the reason Lynch found himself at the Pentagon that day in the summer of 2015. Before moving to Washington, D.C., Lynch knew nothing about the military. The closest he had come to the national-security world was watching Saving Private Ryan and Full Metal Jacket. He does not fit the stereotype of the military man, either. Lynch is 5-foot-9, slim, and in his own words, “very, very average for a human being.” He smiles often and boasts a pair of geometric tattoos, a Fibonacci spiral on his left biceps and a spiro-graph down the length of his right arm. His dog—a miniature pinscher named after the film producer Dino De Laurentiis—is afraid of motorcycles. When I went on a walk with Lynch and Dino in March 2020, Chris wore a black fitted T-shirt, white Ray-Bans, and gym shoes with tie-dye laces. The look spoke more to his background in the Seattle start-up scene than his current role as one of the top technologists in U.S. national security.
Lynch also came in with the skepticism of the government that many in the tech industry share. There is a perception that “you can’t do anything in government because bureaucrats don’t care about technologists … it’s a waste of your talents,” Lynch told me. When a friend of Lynch’s shared that he was going into government through the U.S. Digital Service, Lynch told him flat out, “That is the dumbest fucking idea I’ve ever heard.”
Chris Lynch ran the Defense Digital Service before founding Rebellion Defense, one of a new breed of government contractors. (Courtesy of Christopher Michel).It wasn’t until Todd Park, the White House’s chief technology officer, personally flew to Seattle to recruit Lynch that he was persuaded to join the organization. And after spending a month and a half building file-conversion software for military doctors, Lynch had a change of heart.
“I realized that having a mission is meaningful work,” he said. “Just because a bunch of nerds did the most seemingly simple project, somebody would not potentially die.”
But it also showed just how far the American military had fallen behind the rest of the world in its technology expertise. The same year Facebook released software that could describe images to the blind, the Pentagon needed help converting JPEG files to PDFs. If a small team of programmers could make this much of a difference in 45 days, Lynch thought, something was very wrong.
Lynch was right. Traditionally, a military’s power has been defined by its strength in air, on land, and on sea. For decades, the United States had raised a military that could outmatch any other global fighting force in each of these domains. However, over the past 20 years, we have seen a paradigm shift in the domain of national security. No longer is a country’s military strength defined solely by the size of its fleets, the speed of its vehicles, or the destructive power of its munitions, as it was in the 20th century and every era before. In the 21st century, militaries must also project their strength in a new domain: cyberspace.
Today’s militaries can deal a physical blow with a digital signal. With the right lines of code, you can disable a nuclear reactor, destroy a munitions factory, or knock out power to an entire country. You can infiltrate the computer networks of your enemy, surveil their every move, and stop them from launching attacks on you. The digital warrior never needs to look up from their keyboard.
Cyberweapons are not the only digital technology that is transforming national security. Artificial intelligence is also revolutionizing how militaries do battle and spy agencies conduct espionage. Using AI systems, governments can spot individuals in a crowd, locate facilities to attack, detect intrusions on a computer network, predict civil uprisings, and identify potentially violent extremists.
Although the United States can best any other country in the traditional physical domains, it faces a much more level playing field in cyberspace. The Pentagon has been slower to adopt national-security technologies like artificial intelligence than certain other countries, including China. One major reason is that the leading developers of these new digital tools are not members of the traditional military-industrial complex—companies such as Lockheed Martin, Raytheon Technologies, and Northrop Grumman. Instead, they are in the technology industry.
This is a significant difference. Traditional military contractors have worked for decades in lockstep with the Pentagon. The connection was not just financial; it was political and cultural. Many executives and board members of companies within the traditional military-industrial complex had previous jobs in the military or CIA. “The mission continues” is a typical sign-off in emails between contractors from .com email addresses to their counterparts with .mil and .gov email addresses. But the leaders in the tech industry were people like Chris Lynch, skeptical of government bloat and bureaucracy, whose hackles were raised by the ways that the same technology they were making could be turned toward destructive ends.
In 2015, Lynch became the director of the Defense Digital Service, and in 2019, he left to found his own company, Rebellion Defense. His company is one of several to emerge in recent years to provide countries with the technologies needed to protect national security in the 21st century. This industry includes a handful of big-name players such as Microsoft and Palantir, as well as countless smaller companies that few people outside the national-security world would know, like Rebellion Defense. This new breed of government contractor is changing the way the U.S. and its allies conduct foreign policy around the globe, and they are upending the traditional relationship between sovereign nations and their defense businesses.
Lynch’s overall aim with Rebellion Defense is to provide the U.S. military with much-needed digital tools while also helping young technologists reach the same epiphany about public service that he did. His strategy is encapsulated in the name of the company.
While leading the Defense Digital Service, Lynch fashioned the organization into something of a haven for Star Wars geeks. Team members gave projects names like “Boba,” “AT-AT,” and “Jedi.” The group had an office in Augusta, Georgia, called Tatooine. Lynch’s going-away party from Defense Digital Service was attended by a group of Pentagon staff dressed up as Star Wars characters. During our walk, he showed me a photo from the party that included him, his dad, Chewbacca, General Paul Selva (who was at the time the nation’s second-highest-ranking uniformed officer), and a fully robotic R2-D2.
But what really set the tone for the group was the sign outside Lynch’s office, which read defense digital service, rebel alliance. According to Lynch, the plaque was intended to signify to the team that they were the “rebels” bringing change to the bureaucracy that surrounded them. There is a certain irony in naming a defense-technology company after the destroyers of the Death Star, and Lynch himself is quick to acknowledge it. Still, it has helped bridge the gap between Washington and Silicon Valley, he said. “Who doesn’t want to be part of the Rebel Alliance?”
Lynch also acknowledged that in today’s world, the line between good and evil is much less cut-and-dry than it was a long, long time ago in a galaxy far, far away. Even compared with the Cold War era, the landscape that Rebellion and its counterparts navigate today contains many shades of gray. Arms controls are not nearly as clear-cut for digital technology as they are for old-fashioned ballistics, so modern defense companies have much more that they need to figure out on their own.
Throughout the Cold War, the United States created a broad regulatory framework to limit the export of weapons and national-security technologies to foreign countries. For example, the International Traffic in Arms Regulations prohibits non–U.S. citizens from accessing technical data or physical materials for certain defense-related technologies. Other measures—such as the Arms Export Control Act and Export Administration Regulations—restrict the export and use of national technologies to foreign countries. The goal behind such standards is clear: to prevent national-security technologies developed by the United States from winding up in enemy hands. The government can customize its arms-control agreements to fit its relationships with different countries. The regulations for the United Kingdom differ from those with Turkey, which differ from those with Iran. During the Cold War, these laws cemented the lines between militaries of liberal democracies and communist states, and in the post–Cold War period, they have (mostly) prevented traditional weapons from falling into the hands of terrorists and adversarial nations.
But the arms-control regime of the 20th century largely does not apply to the national-security technologies of the 21st century. One of the main reasons is that digital tools like AI are more difficult to categorize than traditional defense technologies.
Fighter jets and warships are used for one thing: the projection and exercise of military power. But artificial intelligence is a general-purpose technology with both national-security applications and completely benign commercial uses. A computer-vision algorithm can be trained to spot enemy combatants on a battlefield, but it can also be used to tag friends in social-media posts and power self-driving cars. AI takes on the values and intentions of its human masters. The same AI-enabled facial-recognition technology that can identify known terrorism suspects can just as easily profile and track members of an ethnic minority. The technology is also imperfect. The accuracy of artificial intelligence depends on the quality of the data used to train it, and how the software reaches a particular conclusion is not always clear. Although you can tolerate a certain number of errors in a system that provides online shopping recommendations, the consequences of an error on the battlefield can be lethal.
These are core concerns of Rebellion Defense’s work. It makes traditional IT and cybersecurity tools, but its bread and butter is AI. This includes software that can read text, classify images, analyze video, and process the enormous amount of information flooding into the Pentagon from every corner of the globe.
However, when it comes to arms control, the “dual use” nature of artificial-intelligence software creates a conundrum for U.S. policy makers. Subject all AI systems to the same regulations that apply to nuclear warheads, and you stifle innovation and hamper the American tech industry. But leave them completely unregulated, and you could enable terrorists and enemy militaries to get their hands on powerful weapons of war, made in the USA.
To effectively regulate the sale of AI and other emerging technologies, policy makers must first agree on which narrow applications pose a national-security threat if they wind up in enemy hands. But today, the definition of “security-sensitive” technology varies widely based on whom you ask, according to MIT’s R. David Edelman, a former White House senior official who led policy making at the intersection of technology and national security.
“That question about what’s really sensitive is a fundamental debate that is taking place … at the government level, which is not always informed by technology; at the industry level, which is certainly not always informed by government; and at the researcher level, which is sometimes not informed by either of them,” Edelman told me. “You’ve seen little blips where these communities get out of sync.” The blips cited by Edelman can become blowups, as has been the case with AI technologies including drones, autonomous vehicles, and facial recognition.
Edelman continued, “If you were to go ask researchers what constitutes an AI technology, they would give you exactly as many answers as the number of researchers you asked, possibly plus five or six. The reality is that [the label] AI means everything and nothing.”
The confusion is compounded by the lack of technical expertise in the halls of government. Today, the 30 students in my son’s high-school class have far more technological savvy than all but a handful of the 535 members of the U.S. Congress. Informed policy decisions require informed policy makers, and most of the government is still interpreting the national-security challenges of the 21st century through the lens of 20th-century technology.
In January 2020, the U.S. Commerce Department issued its first export-control regulation on an artificial-intelligence system. The rule limits the sale of AI software that can automatically analyze geospatial imagery, ostensibly collected by military drones and satellites. Although this is a significant step, the government did not exactly break new ground. Geospatial technology was already highly regulated. Companies could not sell imagery above a certain resolution, and both drones and satellites are themselves subject to International Traffic in Arms Regulations and other export controls. Policy makers simply amended an old framework to accommodate a new technology.
However, there are many applications of AI for which there is no precedent in the existing arms-control framework. AI systems can help authoritarian regimes consolidate power within their own borders. Although facial-recognition and surveillance technology may not fit the traditional description of national-security technologies, they are no less threatening to free and open societies. Yet Western companies have exported these technologies for years, Edelman said, and “I think most members of the American public and certainly a lot of public policy makers wish they hadn’t.” Beyond this gray area, there are algorithms that can be even more consequential. Today, companies are developing systems that can identify enemy combatants on the battlefield, power semiautonomous weapons, and coordinate drone swarms.
Erik Carter.Edelman explained that certain types of AI can be more easily weaponized than others and that “those are the sorts of implementations that it is entirely appropriate to regulate, and frankly, government’s a little bit behind the ball in identifying them.”
U.S. military leaders have begun to stress the importance of AI ethics, and in 2020, the Pentagon signed on to a set of five broad principles for the ethical application of the technology. However, these principles are vague, and contain platitudes such as that “personnel will exercise appropriate levels of judgment and care” when developing and using AI.
Today’s geopolitical landscape is not as binary as it was during the Cold War, and countries cannot be classified as either “allies of democracy” or “allies of communism.” Political and economic models fall on a spectrum from open to closed, with lots of gradations in between, and national alliances are not as fixed as they once were.
Chris lynch and other members of the cyber-military-industrial complex are navigating this new world largely on their own. This puts them in a position where they need to formulate clear principles for the types of technology they are willing to develop, and what goes too far. This is a lot to ask of a company, and it cannot handle those tasks wholly on its own. The challenge is exacerbated when the technology executive is young and may be a great engineer but does not have much experience in the world of geopolitics. There is a difference between intelligence and wisdom, and I have seen too many mistakes made by technology executives who are very intelligent but not yet wise. I recall an example from my own time in government when a mobile app developed in California became a favorite tool of the Assad regime, which used it to identify political enemies. In another case, a well-intentioned mobile video program unwittingly gave conflict-zone intelligence to militias in the east Congo.
At the same time, when the technology sector has so much more expertise than the traditional defense sector, it is worth harnessing that expertise and ensuring that technology companies shoulder the responsibility for what they are making. A system that allows companies to weigh in and even lead allows more informed innovation and implementation—and provides more checks and balances than a system in which the government decides and drives everything.
For Rebellion Defense founder Chris Lynch, that sense of responsibility is a motivating force. “If you have strong opinions about national defense and security and the utilization of all these technologies that are ultimately going to change the world over the next 50 years, you have an obligation to show up at the table,” Lynch said. “You are providing the things that people need, and you’re helping craft the strategy, the policy, the implementation, and the execution of how those technologies will be used.”
At Rebellion Defense, employees meet once a month to discuss the types of projects and customers the company would refuse to take on. For example, Lynch said, the company has already determined that it will not build domestic surveillance technology, nor will it aid U.S. officials in rounding up undocumented immigrants. Lynch was reluctant to disclose Rebellion’s other lines in the sand, though he said the company has turned down multiple offers based on feedback from employees.
In a world of self-regulation, these decisions and the processes that produce those decisions will vary widely from company to company.
In September 2017, Google began working with the Pentagon on a broad artificial-intelligence initiative called “Project Maven.” Google’s particular project sought to build AI software that could sift through the troves of footage collected each day by military drones. The system would save intelligence officers from the tedious task of analyzing the footage frame by frame. (This is the sort of geospatial-analysis software that would fall under the government’s January 2020 export controls.)
Within months, Google employees began protesting the project, which they argued would help the Pentagon better target its drone strikes. In April 2018, some 3,100 employees signed a letter demanding that Google stop participating “in the business of war.” Soon after, Google declined to renew its contract with the Pentagon.
Chris Lynch disagreed with the decision by Google’s management to give in to employee pressure. As he saw it, Google forfeited an opportunity to directly influence how the Pentagon uses artificial intelligence. Instead, the contract went to Anduril Industries, a defense-technology company co-founded by Palmer Luckey, a controversial libertarian in his 20s who helped invent the Oculus Rift virtual-reality headset.
Anduril was contracted to build an AI-powered sensor network that would provide troops with a virtual view of the front lines. The sensors would be mounted on drones, fixed towers, and troops themselves, and used to identify potential targets and direct autonomous military vehicles into combat. The software helps troops in the field make real-time operational decisions. It might not directly decide who lives and who dies, but it will significantly influence how troops arrive at that answer.
Anduril went on to build a similar AI-sensor network to help U.S. Customs and Border Protection to coordinate operations along the U.S.-Mexico border. When asked in 2018 whether there were any Pentagon projects Anduril would turn down, Luckey punted, saying, “That’s not really totally up to us. We are working with the U.S. government.”
That said, Anduril CEO Brian Schimpf told me there was one thing the company would not do: It would not build systems that execute “lethal force” without a human in the loop. In other words, Anduril will not create robots that can kill on their own accord.
“This is a military decision-making responsibility—it can’t be outsourced to a machine,” he said. “Everything else is one of these questions where I think it’s mostly a matter of the controls on how the technology is employed. There are very few other technology areas that I think have those sort of bright lines.”
Schimpf thinks it is the responsibility of military leaders to set those controls, and he trusts them to make the right call in the end. “Any of these [applications] that are too out-there, they eventually get shut down, they eventually get stopped. The U.S. system may take a while, but it is quite robust to keeping a lot of these overreaches in check.” That all three companies ended up resolving on different principles in the development of AI might seem worrying—but that is also part of the debate that needs to happen with such new technologies. There are no clear-cut ethical answers at the start.
Realistically, the Pentagon does not have much choice in whether to develop its artificial-intelligence capabilities. China and Russia are investing heavily in military AI, and the national security of the U.S. and its allies will suffer if it does not do the same. Russian President Vladimir Putin recently remarked to a group of students, “Artificial intelligence is the future not only of Russia but of all mankind,” and added that “whoever becomes the leader in this sphere will become the ruler of the world.”
For companies that build facial-recognition and other technologies that can empower authoritarian regimes, that also means being responsible about their customer base. Claiming ignorance is no longer a valid justification, said R. David Edelman, the former White House senior official. “It is no longer an acceptable excuse for a tech CEO to say, ‘Well, I didn’t know what use they were going to put it to.’ The sort of near-criminal negligence that we heard from tech CEOs of even three years ago is simply no longer plausible in today’s era.”
Technology companies have the expertise that makes technology applications possible and reliable. But it is also important for technology companies to have the autonomy to decide how their relationship with the national-security community will proceed, and to develop clear principles for how what they build can be used. If they have an objection to specific applications of AI that they feel pressured to develop, it is worth voicing those objections, in a way that other companies and policy makers alike can weigh.
“Now more than ever, we need to bring technologists into a place where they can help shape and craft the policies and the direction of not only how these technologies will be built, but how they will be used,” Lynch said. “If the conversation is only happening in the Department of Defense, that is not a long-term strategy. If the conversation is only being had in a coffee shop in San Francisco with a bunch of people who have never spent a moment thinking about the mission of defense, those people are failing just as much. If you don’t bring those two sides together, there is one thing I am 100 percent certain of, and that is that nobody will be happy with the outcome. If you don’t have that discussion, and if you don’t participate in that discussion, we end up in complete and total failure.”
No comments:
Post a Comment