This Company Is Helping Build China’s Panopticon. It Won’t Stop There

(Bloomberg Businessweek) -- The lobby of SenseTime’s Beijing office makes you feel a bit like you’ve stumbled into a Philip K. Dick novel. A panel near the entrance acts as a digital mirror, its built-in camera analyzing your face to estimate your age and assign an “attractiveness rating” (you score higher when you smile). Another screen shows an app that morphs your face like a Snapchat filter, except that instead of adding rainbow vomit, it slims the figure, widens eyes, and whitens skin.

A third screen displays a feed from a camera aimed at a distant intersection, with every person and object instantly identified and overlaid with a color-coded box. Humans are orange; cars are blue. The longer the system watches, the more bullet points appear beside the boxes: “Adult. Short-sleeve top. Trousers. Male. Black top. Gray bottom,” one set of points reads. “Volkswagen Passat. Black,” reads another. Motion lines trailing each figure indicate their direction and speed.

SenseTime sells artificial intelligence software that recognizes things, especially people. The four-year-old company’s image-identifying algorithms have made it the world’s most valuable AI startup (more than $4.5 billion) and an early leader in China, where it’s won contracts with the top phone makers, the largest telecommunications company, and the biggest retailer. Various Chinese police departments use its SenseTotem and SenseFace systems to analyze security footage and bust suspects. The company’s success is a marker for China’s race to build digital panopticons to keep watch over its 1.4 billion residents, with direct government support and fewer of the privacy concerns that tend to drive the debate around this kind of next-level surveillance.

The company will have to wrestle more seriously with ethical questions as it expands into new industries and countries. These include Japan, where it’s making road-tracking software to help steer driverless Hondas, and the U.S., where its New Jersey health lab is developing cancer detection software. It’s also working to bring interactive games to livestreamers in Southeast Asia, teaming up with the popular app Bigo.

Although SenseTime is less aggressive than local rival Megvii Inc. and some other security companies when it comes to pursuing government contracts, such business represents about two-fifths of its revenue, according to Chief Executive Officer Xu Li. He says SenseTime’s AI systems are more mindful of privacy than old-fashioned cameras because, for the most part, the data collection is passive, with machines doing the watching. “It actually prevents people directly looking at those kinds of video streams,” Xu says.

This Company Is Helping Build China’s Panopticon. It Won’t Stop There

Although that’s beside the point of privacy advocates’ concerns, critics’ questions haven’t slowed Xu down. Backed by investors including Alibaba, Fidelity International, Qualcomm, Silver Lake, and Tiger Global Management, SenseTime is adding to its staff of 2,200 as it pitches a host of industries on both sides of the Pacific. It expects to triple revenue this year, to $300 million, say two people familiar with its finances. The company declined to comment.

Beyond spotting and ranking faces in a crowd, SenseTime’s long-term goal is to put its technology at the core of image recognition everywhere, from security and shopping systems to personal ID services that work by scanning faces. The idea can be seen in microcosm at its Beijing office, where staffers never swipe a badge to enter. A glass door simply slides open after a hidden camera checks their faces against company records. “That’s really how they see future interactions,” says Jean-François Gagné, who runs Canadian startup Element AI Inc. “You don’t need to log in to your computer, you don’t need to get a boarding pass, you don’t need to do anything anywhere. You’re just recognized.”

SenseTime’s embrace of government work and everyday surveillance contrasts with the present tone in Silicon Valley. Earlier this year, irate Googlers forced their employer to abandon a Pentagon project and pledge to avoid working on autonomous weapons that use AI to fight human wars, staff at Microsoft and Salesforce.com railed against contracts with federal agencies, and Amazon.com employees protested the sale of facial recognition gear to police forces. (Amazon reportedly pitched the tools to U.S. immigration authorities.)

On Oct. 31, Chinese President Xi Jinping and the Politburo held a study session to emphasize the importance of taking a global lead in AI development by boosting investment in technologies and training more professionals, according to state news outlet Xinhua. At the event, SenseTime co-founder Tang Xiao’ou, also a lecturer at the Chinese University of Hong Kong, called for closer ties between state-owned enterprises and private companies, as well as international cooperation.

Washington is paying attention. A congressional report in September warned that China could overtake the U.S. in AI capabilities and raised concerns about an “AI arms race” that could sideline ethics and discretion. Among AI researchers and investors, the conventional wisdom is that while the U.S. and Europe remain much better at developing AI tools and breakthroughs, China is the country that best uses and improves them by feeding the algorithms huge sets of data generated by its massive population. Chinese AI companies excel at “scaling and deploying technology faster than anybody else,” says Aydin Senkut, a managing director at Felicis Ventures LLC.

Outside China, the nation’s surveillance specialists face scrutiny related to discrimination against minority populations. Global human-rights groups have criticized China’s treatment of Muslim minorities in Xinjiang, a western province where more than 1 million ethnic Uighurs are said to be held in detention. (China says they’re undergoing “vocational training” to prevent radicalization.) SenseTime set up a joint venture called Xinjiang SenseTime Leon Technology Co. in the province last year, but the company declined to say whether its technology is at work in the network of cameras that scan many license plates and faces at checkpoints. The authorities’ system has clearly been enhanced by some company’s powerful AI software, says Adrian Zenz, a researcher who studies government contracts in the region. “It’s become a real key component of the security apparatus,” he says. “The point is to reliably identify a person based on their biological properties.”

In September, China named SenseTime one of five “national champions” in AI alongside internet powerhouses such as Alibaba Group Holding Ltd. and Tencent Holdings Ltd., a distinction that means critics and U.S. officials will be watching its expansion more closely. But at the Singapore Defense Technology Summit this summer, co-founder Tang stood before more than 400 military and government officials and contractors from all over the world and said SenseTime doesn’t have the luxury of worrying about some of AI’s moral quandaries, including doomsday scenarios of machine uprisings. “We’re not really thinking very far ahead, you know, whether we’re having some conflicts with humans, those kinds of things,” he said. “We’re just trying to make money.” —With Gregor Stuart Hunter

To contact the editor responsible for this story: Jeff Muskus at jmuskus@bloomberg.net, Jillian Ward

©2018 Bloomberg L.P.