How One AI Startup Decided to Embrace Military Work, Despite Controversy

(Bloomberg) -- A few years ago, Clarifai, a five-year old startup that makes image-recognition software, held a discussion among its workers about two kinds of controversial business partners. The first was the pornography industry, where several companies had expressed interest in automated analysis that would classify videos according to the specific acts in each scene. The second was the military. At the time, Clarifai decided to avoid both kinds of contracts, according to three people with knowledge of the conversation.

Clarifai continues to bat away the smut peddlers. But Matt Zeiler, its founder and chief executive, is now embracing defense work. On Thursday, the New York-based company is announcing the formation of a subsidiary in Washington, D.C. called Neural Net One, which will exclusively handle government work. It will pursue defense and intelligence contracts, as well as work in disaster relief, border security, and environmental monitoring. 

The opening of Neural Net One comes after the Department of Defense last summer hired Clarifai to participate in Maven, a project the government said would “leverage advanced commercial technologies to provide advantage to the warfighter in contested environments.” The Pentagon allocated $16 million for Maven in fiscal 2018, and $13.9 million in fiscal 2019. Alphabet Inc.’s Google has also  been involved. Maven was significantly larger than any of the Clarifai’s deals with commercial clients, according to three people familiar with the project. They asked not to be identified for fear of retaliation.

Clarifai’s work on Maven was controversial internally. Four former employees said Zeiler’s lack of candor about the project damaged morale, complicated recruitment, and undermined trust within the company. At least two employees left Clarifai because they were concerned that its focus on military work would subsume the company’s other work. “If you’re a Palantir, and that’s the company culture, that’s fine,” said one of them, referencing the company that Peter Thiel founded in 2004 to work with the Pentagon and Central Intelligence Agency in war zones. “But the company culture here was kind of the opposite.” 

The discontent at Clarifai mirrors the response to military and law enforcement contracts at larger technology companies like Google, Microsoft Corp., and Amazon.com Inc. At those companies, recent revelations of government contracts have inspired employees to demand the companies stop pursuing deals for the military or law enforcement. Over 3,100 people and 244 organizations working on AI have signed a pledge not to create automated weaponry. 

Zeiler acknowledges that early discussions about whether to work on various applications had taken place, but said he never ruled out working with the military. “As we evolve as a company, we continue to revisit opportunities, markets and technologies so that we can tackle problems and innovate on accelerating the progress of humanity,” he wrote in an email. Zeiler said that staffing issues were a reason to open a subsidiary that operated autonomously, adding that the company has spent the last several months developing an ethics department. 

Zeiler said in an interview that he always saw the Maven contract as a tryout with the Defense Department, and that government contracting “has the potential to grow faster than the commercial side.” There are more than two-dozen active solicitations for federal contracts listing computer vision or image processing in their requirements, most of them for work at the Defense Department, according to Chris Cornillie, an analyst with Bloomberg Government.
 

Matt Murphy, a Clarifai investor who also sits on its board, framed the new focus in loftier terms. “From a patriotic standpoint I think it’s a good thing,” he said.

Zeiler founded Clarifai in 2013, and clients began using Clarifai’s technology to comb web forums for inappropriate imagery, or sort items in online stores. But most of its contracts were small, and only a few customers made up the lion’s share of its revenue. Sales were slow last year, said someone with knowledge of the company’s finances. The company also lost several key employees, including its vice presidents of product, engineering, and research, the latter two of whom each lasted only several months at the company. 

The company has raised $40 million, and Zeiler downplayed Clarifai’s challenges. He  said it doesn’t have cash flow problems, and has had some recent months of profitability. Still, 2017 was a time that Clarifai could have used a sense of validation, and the Department of Defense showed up to offer it with perfect timing. “It was cool too see a lot of interest into Clarifai being a leader on AI,” said Zeiler. When the company took on Maven, he offered people who worked on the project financial incentives, and warned them they might have to work through holidays. About 10 people were involved. 

At first, Zeiler didn’t fully explain what the new project was, even to those working directly on it, according to three former Clarifai employees. That was unique among Clarifai’s contracts, they said. But there was a feeling internally that the new work was significant. One former employee called it a lifeline for the company, a characterization Zeiler disputed.

But as it became clear that Maven was part of a military application, discomfort grew.  Some workers resented how the company had acted without re-opening the discussion with employees. Even those who objected to Maven said it was hard to say how many people opposed the project. Zeiler said two people asked to be transferred off the project, with one citing a personal objection to working with the government.  

In June, Amy Liu, a Clarifai employee working on Maven, filed a complaint with the Department of Defense alleging that the company had suffered a security breach. She then made the company’s work on Maven public in a subsequent interview with Wired. According to Liu, the contract was worth $7 million for six months. (Clarifai declined to comment on the size of the contract.) Liu declined an interview request, saying she had reached a settlement that forbade it. 

Zeiler at first declined to confirm to Wired that Clarifai was even involved in Maven, before deciding to post a blog entry defending its work the next day. He also disputed Liu’s allegation that Clarifai had suffered a security breach. 

In May, Clarifai hired Sean Alger, who had a history selling software to the government, to build what became Neural Net One. The office has a staff of 11 people, accounting for over 10 percent of Clarifai’s workforce. In an interview, Alger spoke extensively about how Clarifai’s technology could be used to help find survivors in disaster zones, identify missing children, and monitor the impacts of climate change. When the conversation turned to military applications, he became vague. “We have solutions around aerial and satellite imagery, that detect objects and the type of object,” he said. “Absolutely.” Zeiler and Alger declined to predict how much of the company’s overall business could eventually be dedicated to government contracting.

Zeiler said he expected controversies like the one over Maven to become less common as people accept its utility. “What happens when someone steals your kid when you’re at the grocery store? Would you want the best technology to find your kid who’s now missing, or would you want to rely on stuff from the 70s and 80s and a lot of manual work?” he said. “It’s a tradeoff.” 

Zeiler and Alger both said that Clarifai’s staff has become more comfortable with Maven as they learned more about it. Alger cited an internal poll showing a positive response, although he didn’t provide specific metrics.  At the same time, worker revolts against military contracts at other companies might actually benefit Clarifai, said Alger.  “One of my largest competitors is Google,” he said. “If they want to back out of a market I imagine that only increases my chances of success.” 

To contact the editor responsible for this story: Emily Biuso at ebiuso@bloomberg.net, Andrew Pollack

©2018 Bloomberg L.P.