It has become fairly cliché to call China’s surveillance state—its artificial intelligence-driven facial recognition, the new “social credit system,” its cultural policing and re-education camps for Uyghur minorities—“something right out of George Orwell’s Nineteen Eighty-Four.”
But that doesn’t mean it’s not true.
Orwell’s dystopian vision, first published 70 years ago this June, was informed by the fascist and communist movements that triggered worldwide military conflict and the deaths of millions of people during the mid-20th century. But Orwell’s warning went well beyond the wars we knew. It cautioned, noted Erich Fromm in an afterword in the 1961 edition, against the loss of humanity and free thought, and the new, increasingly centralized “managerial industrialism, in which man builds machines which act like men and develops men who act like machines.” It showed how that could be used as a tool of totalitarian ambitions.
China convulsed in revolution during the 1940s; Mao Zedong established the one-party state of the People’s Republic of China (PRC) in 1949. But Orwell, according to Fromm, had his gaze on Joseph Stalin and the Soviet Union when he wrote Nineteen Eighty-Four, which is set in fictional Oceania, one of three of the world’s superstates. Oceania is run by one party, Ingsoc (English Socialism), headquartered in what is left of post-war London. “Big Brother” is the god-like, mustachioed visage serving as the party’s fearsome symbol of absolute authority.
As rooted as Nineteen Eighty-Four is in Orwell’s own uncertain world, he could not have imagined how his predictions for humankind would morph and metastasize beyond the terrors of Mao’s Cultural Revolution and into the cyber-authoritarianism we are seeing in Xi Jinping’s China today (still run by the Chinese Communist Party).
Where the Soviet experiment failed, China’s has evolved more powerfully into a hybrid of state-run capitalism. Control is maintained through coercion and social management. There is no rule of law as it is universally understood, and elections are compulsory, with all candidates, from the most local to the top, pre-approved by the party. Only sanctioned religious worship is allowed. Private enterprise is only “free” by courtesy. Successfully petitioning the government is nearly impossible, if not dangerous.
The Internet of course is strictly censored. While one can buy a copy of 1984, any reference comparing it to modern authoritarian governments, especially Maoism, is forbidden.
“The Chinese Communist Party does not say ‘war is peace,’ but it does claim to care about ‘democracy,’ while denying space for competing views to be expressed in public,” Jeffrey Wasserstrom, a professor of history at the University of California at Irvine, tells TAC, drawing a line from Orwellian Newspeak to language published in Chinese Communist Party (CCP) manifestos.
Furthermore, he compares the “stamping out of any discussion” about the Tiananmen Square massacre 30 years ago to the “memory hole” through which pages of undesirable history are tossed by worker drones in Ingsoc’s Ministry of Truth. Last year, the CCP announced a takeover of all mass media regulation by China’s Central Propaganda Department. Major broadcasters, in part, will be responsible for “propagating the party’s theories, directions, principles and policies.”
So it should come as no surprise that citizens are now pressured to spend more and more time on the “hottest app” in China—“Study the Great Nation”—a bible of sorts highlighting the teachings and goings-on of president Xi Jinping. Individual participation rates are recorded, and those who don’t spend enough time on the mobile app everyday are punished by employers and chastised by school teachers.
“He is using new media to fortify loyalty toward him,” Wu Qiang, a political analyst in Beijing, told The New York Times.
Outside observers say say Xi’s consolidation of power within and for the party reflects the need to minimize political and economic rancor at home, and position China as a unified power player abroad. Indeed, the government seems to find new ways to use technology to manifest this power every day. Ironically—and Orwell himself may have never dreamed this—is that thanks to globalism’s successes, much of that technology is coming from us.
“Certainly the social credit, the total face recognition systems, the concentration camps in China, evoke all the dark imaginings of a 20th century dystopian like Orwell,” Adam Simon, a Los Angeles-based science fiction screenwriter, director and producer, tells TAC.
“But what is perhaps most pernicious is how many of the Chinese techniques are created and enabled directly and indirectly not by some nefarious ‘communist’ social weapons lab or politburo but by the ‘best of the West’ from London to Silicon Valley.” (Think Microsoft, IBM, Google.)
Repeated attempts by TAC to reach officials at the Chinese embassy in Washington for comment on this story were unsuccessful.
All Seeing Eye
For Winston Smith, Nineteen Eighty-Four’s doomed protagonist, his inner thoughts, a running conversation with himself, were the only thing Big Brother wasn’t watching. Yet.
The moment he put pen to paper in a secret diary, written just out of sight of the telescreen dual-functioning as a surveillance camera in his London flat, he considered himself “a dead man.” Sadly, he wasn’t far off base.
In his one-room apartment, there was “no way of knowing whether you were being watched at any given moment” through the telescreen. “How often, or on what system, the Thought Police plugged in on any individual wire was guesswork,” Winston surmised. “It was even conceivable that they watched everybody all the time.”
Outside, cameras and microphones picked up everything. The Police Patrol, hovering in helicopters, were forever “snooping into people’s windows.” In the Ministry of Truth, the propaganda nerve center at which Winston worked, telescreens served the same purpose—in the cubicle, the canteen, the restrooms.
In China today, there are more than 200 million cameras and an unknown number of security robots roaming the streets, watching the populace, in public and private spaces. But unlike Oceania, these “telescreens” are equipped with artificial intelligence, including the ability to identify faces and match them with massive amounts of personal data already harvested by the state.
These cameras are not only used to catch crooks and identify potential terrorists, but to predict when someone may commit a crime, which sounds more like Minority Report than Nineteen Eighty-Four. They also monitor more mundane social transgressions, like public intoxication, jaywalking, or using too much toilet paper. Culprits are publicly shamed, or worse, their social credit score goes down and they’re blacklisted (more on that below).
Right now, facial recognition is feeding a national database that, according to reports, is striving to identify any one of China’s 1.4 billion people within three seconds. Research firm IHS Markit estimates that about 450 million new cameras will be shipped to the Chinese market by the end of 2020. China certainly has more cameras per person than any other country in the world (though with more than six million cameras, Britain is not far behind).Facial recognition is already creeping into police work in the U.S (some estimate 50 million cameras here), but localities across America are starting to ban its use. There is no such luxury of protest in China.
In May, The New York Times reported that the Trump administration is considering limits on the ability of Chinese-run Hikvision to buy American components for its video surveillance technology. The report said Hikvision tech already boasts the ability to “track people around the country by their facial features, body characteristics or gait, or to monitor activity considered unusual by officials…”
While there was no internet, no cloud, no algorithms in Nineteen Eighty-Four, Winston always feared that his facial expression and body language would seal his doom.
“It was terribly dangerous to let your thoughts wander when you were in any public place or within range of a telescreen,” Winston explains. “The smallest thing could give you away. A nervous tic, an unconscious look of anxiety, a habit of muttering to yourself—anything that carried with it the suggestion of abnormality, of having something to hide.” Not showing the proper joy when it was warranted, or hate when it was called for, could be punished as a facecrime.
Not only used at ATMs, facial technology is now deployed by banks in China to gauge whether a person is honest when they are applying for a loan. Ping An, a financial services conglomerate, has developed software for its banks that “can pinpoint 54 brief, involuntary micro-expressions that a face often creates before the brain can control the movements of the face.”
Because this technology is deployed everywhere—people are now “paying with their face” at a growing number of retail chains and travel stations—the government has access to broader and more complex data streams: where individuals shop, how much debt they carry, where they socialize. Everyone’s story is told with one capture of their face (made easier now, as police in select areas have sunglasses that can call up tons of data on a person on the spot).
To grasp the full realization of this new technology, one need look no further than how the country’s 11 million minority Muslim Uyghur population is being tracked.
According to reports, facial recognition is being used to racially profile Uyghurs based on their distinct facial characteristics. Surveillance cameras have been placed in individual homes in Xinjiang, the autonomous region where the biggest concentration of Uyghurs live. There, according to Human Rights Watch, (HRW), an Integrated Joint Operations Platform is being used by police to aggregate “data about people and flags to officials those it deems potentially threatening.” That information includes personal behavior and relationships, the location of personal phones and vehicles, buying habits, and more. All is fed through a database that can both monitor and “predict” threats.
“[These findings] shed light on how mass surveillance functions in China,” warns HRW. “While Xinjiang’s systems are particularly intrusive, their basic designs are similar to those the police are planning and implementing throughout China.”
“A Boot Stamping on a Human Face”
“The innovation now of course is that the technology—above all the algorithms—obviates the need for human collaborators,” says Adam Simon. “It’s the driverless-car version of surveillance.”
Simon is right about how social control will work for China’s massive, sprawling population in the future. But in Xinjiang, “human collaborators” are still at work—“re-educating” an estimated one million Uyghurs who have reportedly disappeared into heavily fortified concentration camps there in the last three years. In addition, a million Chinese nationals have been sent to live with Uyghur families to serve as their cultural minders and as spies for the PRC.
For its part, the government, which has severely restricted outside access to Xinjiang, has said that its tracking and detention of Uyghurs, as well as its crackdown on their Muslim culture, is a matter of national security and “unity.”
Since the 9/11 attacks on the United States, critics say the Chinese have used terrorism to justify their repression of the Uyghur population. In 2017, public intellectuals, teachers, and other high-profile Uyghurs began vanishing. Several, it turns out, were convicted of “separatism” and sent to secret prisons. For all others, their whereabouts are a mystery. Reports over the last year indicate that tens of thousands are in the so-called “education conversion centers,” where, according to the few who have made it out, detainees are beaten, brainwashed, and broken. The goal: total “transformation,”much like Winston at the end of Nineteen Eighty-Four. After his interrogator O’Brien said that the future was a “boot stamping on a human face—forever,” Winston was sent to torturous “re-education” in the bowels of the Ministry of Love, after which he ultimately “loved Big Brother.”
“The Chinese government is committing human rights abuses in Xinjiang on a scale unseen in the country in decades,” charged Sophie Richardson, China director at HRW, in 2018. The group described wholesale “political indoctrination, collective punishment, restrictions on movement and communications, heightened religious restrictions, and mass surveillance in violation of international human rights law.”
In Nineteen Eighty-Four, Winston describes a world in which any transgression against the state’s arbitrary rules or codes of behavior could mean one was “vaporized” or completely “annihilated” from existence. Some were sent to camps. “In the vast majority of cases there was no trial, no report of the arrest,” Winston said. “People simply disappeared, always during the night.”
Kamatürk Yalqun last saw his father, Yalqun Rozi, when he visited his family in Washington in 2015. They later learned that Rozi, chief editor of middle school textbooks in Xinjiang, was sentenced to life in prison by the Chinese government on charges that he was a separatist.
Kamatürk told TAC that after his disappearance and other editors’ for the school system, the PRC cracked down on all the textbooks. Now they are all written in Chinese, and the students forced to wear Chinese school uniforms—no Muslim dress allowed. In the streets, there is a battle over the dress code, with reports that Chinese authorities are keeping women with hijabs and men with beards off public buses—even detaining them—and cutting women’s skirts if they are too long. People are reportedly arrested for reading Muslim books and praying.
“[Uyghur culture] is as different as American culture is to Chinese culture,” and the people are understandably resistant, said Kamatürk, standing in front of a life-size photo of his dad, from whom he has had no word. “My father was the harbinger,” he said. “The true number of deaths in the camps, or died immediately after release, is unknown, given the veil of secrecy and fear.”
When the Chinese government finally broke its silence about the detention centers, officials said they hosted a “vocational education and training program” in which the “students” were treated humanely and “according to the law.”
“Its purpose is to get rid of the environment and soil that breeds terrorism and religious extremism,” exclaimed Shohrat Zakir, chairman of Xinjiang’s government, in October. He added that while some may “graduate” from “deradicalization” successfully, “the duration, complexity and intensity remain acute, and we must maintain high vigilance.”
Just as disturbing are the million mostly Han Chinese nationals who have been sent to Xinjiang in three waves since 2014 to assimilate the Uyghurs. The PRC actually calls them “relatives.” A statement in December 2017 called the project “United as One Family.”
The reality is much darker. These “relatives” are billeted in homes armed with “gifts” of food and appliances and a better understanding of how to behave in the Chinese way. Some live with their “families” for a year or more. They serve as spies, minders, and tutors—in other words, social engineering on a scale not seen since Nazi Germany sent 200,000 Polish children to live with German families to be “Germanized” between 1939 and 1944.
Family and community are the last line of resistance to the state. “Only China can do this because they have the manpower and they have the will to do it. They are like robots. They do what they are told,” said Turdi Ghoja, a Uyghur expatriate in Washington who has not heard from anyone in his family since late 2017.
“They [“relatives”] have absolute power, they are terrorizing Uyghurs,” he tells TAC. “In my mind it is even worse than the concentration camps.”
The last time Ghoja spoke to his brother-in-law, the man was arrested shortly afterward. Ghoja has no idea if there are “relatives” in the home preventing communication, or if his mother and sister have been taken away, too.
“They know everything you say, they monitor all interactions,” Ghoja said. “Sometimes I feel that not knowing is better. Maybe they are in the camps, maybe they are not. I just keep calling, and no one picks up the phone.”
Social Credit System
According to various reports over the last two years, the PRC has rolled out several social credit pilot programs in which individuals and institutions that are deemed “trustworthy” are given positive points, and therefore rewards, while those who transgress lose points. A super-low rating can lead to being “blacklisted,” blocking one’s access to health clinics, private schools, plane and train travel, and jobs.
“At its core, the system is a tool to control individuals’, companies’ and other entities’ behaviour to conform with the policies, directions and will of the [Chinese Communist Party],” writes Samantha Hoffman, a visiting fellow at the Australian Strategic Policy Institute. And government is using all the artificial intelligence described above to do it. “It combines big-data analytic techniques with pervasive data collection to achieve that purpose.”
Some 43 cities are now testing some form of social credit, while a national system is expected to roll out by 2020 to “put people first, broadly shape a thick atmosphere in the entire society that keeping trust is glorious and breaking trust is disgraceful,”according to early planning documents.
Each pilot is different but typically gives an individual 1,000 points to start. Some depend on electronic data streams coming from camera surveillance, banks, and public institutions to take away points for myriad reasons—everything from evading taxes and playing too many video games to violating traffic laws. On the other hand, giving to charity, donating blood, and paying bills on time help increase points.
In places like Jiakuang Majia village everything is done with a designated official spending her day checking in on neighbors and filing reports by hand to the state. Every month she tallies up scores on a single sheet and assigns “little red stars and flags” by villagers’ names on public bulletin boards. In the cities, individuals can check scores on their mobile devices.
Good behavior is rewarded mostly with streamlined services like getting in a faster queue at the hospital, not paying a deposit on rental cars, or better foreign exchange rates. Bad behavior—which can include protesting and petitioning the government—can leave one locked out of public services and social media. According to reports, more than 11 million people have already been prevented from taking flights or high-speed rail (the difference between a three-hour or a 30-hour trip) based on bad social credit scores.
The government says this is making not only individuals but businesses more trustworthy. Companies on the blacklist, for example, are prevented from bidding on government projects or issuing corporate bonds.
Meanwhile, private financial companies are developing their own credit regimes outside of the government using the popular QR code payment systems. In China, almost everything is paid for by scanning a barcode via mobile app, like WeChat, Alipay, or Tencent. Individuals can also use QR codes for identification, tracking family and pets, charitable giving, or posting to job boards. Companies are collecting all that info to build profiles and assign scores.
“What’s troubling is when those private systems link up to the government rankings—which is already happening with some pilots,” says Mareike Ohlberg, research associate at the Mercator Institute for China Studies. “You’ll have sort of memorandum of understanding like arrangements between the city and, say, Alibaba and Tencent about data exchanges and including that in assessments of citizens.”
Surveys indicate that the vast majority of the Chinese people are in favor of this, particularly among the more privileged classes. According to a September 2018 survey by the Mercator Institute, “wealthier, higher educated, urban respondents” viewed it “as an instrument to close institutional and regulatory gaps, leading to more honest and law-abiding behavior in society, and less as an instrument of surveillance.”
This buy-in ensures it will be self-enforcing and public protest will be at a minimum. So when a government petitioner lost 950 points and plummeted to a “D” rating for supposedly writing too many online letters on behalf of his mother’s long held medical dispute with the state, no one seemed to care. When investigative reporter Liu Hu was blacklisted and completely cut off from travel and social media for the crime of being “dishonest,” only the Western media took notice.
“Their eyes are blinded and their ears are blocked,” Hu said of his fellow Chinese people. “They know little about the world and live in an illusion.”
In China, technology has created a system of both commercial rewards along with state controls. Most people say the surveillance keeps them safe and the credit system keeps them trustworthy. Perhaps this is where the comparisons to the bleak and relentless “boot in the face” end, and the iron fist in the velvet glove of Aldous Huxley’s Brave New World (1932) begins, says Adam Simon.
“(Huxley) probably came closer to pre-visioning the seemingly non-coercive, but ultimately totalitarian technologies that simultaneously watch and entertain, crush and seduce today.”
Crush seems to be a key word here. In the end, Hoffman says, the government is using societal problems to justify this authoritarian project, even though “[it’s] a state driven program designed to do one thing, to uphold and expand the Chinese Communist Party’s power.”
Kelley Beaucar Vlahos is an executive editor at The American Conservative. Follow her on Twitter @Vlahos_at_TAC.