American companies continue to build surveillance tools that are used to violate human rights. Workers who refuse to comply deserve protections.
“We can forgive your politics and focus on your technical contributions as long as you don’t do something unforgivable, like speaking to the press.”
This was the parting advice given to me during my exit interview from Google after spending a month internally arguing, resignation letter in hand, for the company to clarify its ethical red lines around Project Dragonfly, the effort to modify Search to meet the censorship and surveillance demands of the Chinese Communist Party.
When a prototype circulated internally of a system that would ostensibly allow the Chinese government to surveil Chinese users’ queries by their phone numbers, Google executives argued that it was within existing norms. Governments, after all, make law enforcement demands of the company all the time. Where, they asked their employees, was the demonstrable harm?
But the time has passed when tech companies can simply build tools, write algorithms and amass data without regard to who uses the technology and for what purpose.
Complaints from a single rank-and-file engineer aren’t going to lead a company to act against its significant financial interests. But history shows that dissenters — aided by courts or the court of public opinion — can sometimes make a difference. Even if that difference is just alerting the public to what these companies are up to.
Nearly a decade ago, Cisco Systems was sued in federal court on behalf of 11 members of the Falun Gong organization, who claimed that the company built a nationwide video surveillance and “forced conversion” profiling system for the Chinese government that was tailored to help Beijing crack down on the group. According to Cisco’s own marketing materials, the video analyzer — which would now be marketed as artificial intelligence — was the “only product capable of recognizing over 90 percent of Falun Gong pictorial information.”
Despite the court’s acknowledgment that Cisco built “individual features customized and designed specifically to find, track and suppress Falun Gong,” several early rulings went against the plaintiffs. And the case is still pending before the United States Court of Appeals for the Ninth Circuit.
The failure to punish Cisco set a precedent for American companies to build artificial intelligence for foreign governments to use for political oppression. This year, an investigation by The Times found that an American company, Thermo Fisher, sold DNA analyzers to aid in the current large-scale domestic surveillance and internment of hundreds of thousands of Uighurs, a predominantly Muslim ethnic group, in the region of Xinjiang. After the story broke, the company said it would no longer sell equipment in Xinjiang.
Public shaming has a mixed track record. In October 2005, five years before being awarded a Nobel Peace Prize, Liu Xiaobo published an open letter to Jerry Yang, the chairman of Yahoo at the time. It was in response to Yahoo’s role in the arrest of the pro-democracy journalist Shi Tao, who had anonymously posted instructions from the Chinese Communist Party insisting that he not report on the 15th anniversary of the 1989 Tiananmen Square protests.
Mr. Liu’s letter said that Yahoo “complied with China’s State Security police by tracing Shi Tao’s internet activity and providing his email account, IP address and other personal information to them. This information became one of the most important pieces of evidence in the conviction of Mr. Shi.” And he further implied that Yahoo’s recently completed deal to purchase 40 percent of Alibaba, the e-commerce giant, was a factor in the decision.
Mr. Liu couldn’t have possibly known that a decade later, Google’s Dragonfly modifications of Search would censor information on his Nobel Prize and reportedly tie Search queries to phone numbers.
Two years after Mr. Liu’s letter, while Shi Tao was serving a 10-year sentence that would ultimately be reduced to eight and a half, Mr. Yang appeared before the House Foreign Affairs Committee. The committee’s chairman, Tom Lantos, berated Mr. Yang’s contention that Yahoo was complying with ordinary law enforcement. “While technologically and financially you are giants, morally you are pygmies,” Mr. Lantos said.
Mr. Yang defended Yahoo’s human rights commitments and emphasized the importance of the Chinese market. Google used a similar defense for Dragonfly last year. Despite negative press received by Google, direct calls from Vice President Mike Pence to end the project and two congressional interrogations of executives, the only major setback to Project Dragonfly came from Google’s privacy team standing up to management.
Collective worker action has been a constant, if unappreciated, check on questionable projects at Google. The trend arguably began when a group of engineers (the “Group of Nine”) prevented the acceptance of an Air Force contract by refusing to build “air gap” technology needed for federal security requirements. This inspired the sustained, and ultimately successful, internal effort to end Google’s work on applying artificial intelligence to Pentagon drone footage for targeting insurgents as part of Project Maven.
The internal turmoil led to the creation of Google’s A.I. Principles, which committed the company to not “design or deploy” technologies that violate “widely accepted principles of international law and human rights.” The combination of Maven’s cancellation and the inability to complete the required federal certifications led to Google’s withdrawal from a $10 billion contract to build the Pentagon’s cloud computing effort called Project JEDI.
The high-water mark of collective action at the company was on Nov. 1, 2018, when 20,000 Google workers collectively walked out of their offices to demand improvements to Google’s policies on sexual harassment. The event was a direct response to a New York Times exposé on Google’s having given Andy Rubin, the “father of Android,” a $90 million payout despite having a verified sexual misconduct claim against him. The action led to public concessions on forced arbitration, not only from Google but across the tech industry.
In Maven and JEDI, internal organizing was a success. The A.I. Principles were drafted and cited as the reason for the cancellations. But even the collective action of key employees on Dragonfly has not led to a public statement that the project would have violated Google’s commitment to not design technologies violating human rights.
Direct action from tech workers has been undeniably effective. Human rights organizations must therefore continue to advocate the legal protection of whistle-blowers and conscientious objectors, including protecting the organizing required for an effective collective action. Further, the broader civil society could increase the frequency of whistle-blowing by creating a dedicated legal defense fund.
If it is morally defensible, tech companies should have nothing to fear from discussions of the human rights implications of their work, whether that discussion happens in the boardroom or public square.
Jack Poulson is the founder of Tech Inquiry. He was previously on the mathematics faculty at Stanford. Mr. Poulson is a former research scientist at Google.
Take Action 
