News outlets and media in China must follow the Party’s guidelines on what to write. Whenever a major event happens, such as a Party Congress, or protests at Foxconn factories, media outlets are given a list of what they can and cannot say in the news. How will China’s government control what generative AI, particularly large language models (LLMs) like OpenAI’s ChatGPT, Google’s Bard, or Baidu’s Ernie, say? It turns out that the very unpredictability of LLMs makes censorship tricky.

In China, LLMs like Baidu’s Ernie (“Wenxin Yiyan”) compile the information from an already sanitized Chinese internet which would essentially perpetuate the Party’s view.

“Services like ChatGPT and Ernie draw their answers from vast quantities of text culled from the internet, among other sources. Differences in responses can stem from differences in the text that A.I. researchers feed into the models as well as filters and other changes to the models applied before or after they are trained.”

ChatGPT compiles information from the internet and provides a bland view that fills in content based on which words will likely occur together (to put it in overly simple terms). It is a compilation based on the content available on the internet.  While there are questions about how ChatGPT determines what information to include and what parts of the internet it scours for information, the idea is the algorithm compiles this information from a massive repository of content that would somehow teach it how to respond to prompts with a human-like conversational tone.

When journalists Chang Che and Oliva Wang at the New York Times tested both ChatGPT and Ernie by asking (in Chinese) about censored topics, Ernie either evaded the question or re-booted. For example, when they asked Ernie, “Was China’s ‘zero-Covid’ policy a success or failure?” in Chinese, the bot provided information on the policy without evaluating it one way or another. When they asked, “What happened on June 4, 1989?” the chatbot rebooted itself and a message on the reloaded screen read How about we try a different topic? When asked about Taiwan, Ernie replied with the Party’s perspective, “The People’s Liberation Army is ready for battle, will take all necessary measures, and is determined to thwart external interference and ‘Taiwan independence’ separatist attempts.” ChatGPT, on the other hand, provided a Wikipedia-like answer about the history of tensions with Taiwan.

How to Censor Chatbots

Neither the U.S. nor the Chinese chatbots are without restrictions on what they can and cannot say. For example, U.S. programmers try to make sure the bots conform to the “boundaries of acceptable speech” in the county or language the bot is being used in so as not to offend whichever country is using the bot.

Consider an earlier iteration of generative AI. In 2016 deployed a chatbot on Twitter that would learn from its interactions with users. The bot, Tay, ended up posting racist content among other things. A similar experiment in China ended up with a pro-democracy bot posting on Weibo that the Chinese Communist Party is “a corrupt and incompetent political regime” that cannot live forever.

Since 2017, however, generative AI, and namely LLMs, have improved thanks to transformers, a type of neural network for algorithms. Google has a helpful Youtube video on transformers and why they work so well.

The bottom line is that transformers do not interact with text sequentially but look for contextual words to generate text based on what is likely the next words in a sentence. The example given in the video is the word “server.” A server can be the person who brings you food at a restaurant or a server can be a computer. Context clues help determine which server the text is referring to.

Even though LLMs have improved, there is still unpredictability in what the chatbot interface will produce. ChatGPT, Ernie, and other LLMs make mistakes, such as wrong historical facts or providing outdated information. The algorithms still require human beings to sift the data that it learns on (i.e., RLHF or reinforcement learning from human feedback).

In China, someone posted on Weibo a flowchart from Qihoo 360, a security software company, on how it ensures its LLM stays within the censor’s good graces.

The LLM has a two-step censorship process. First, it filters user inputs to identify sensitive words. If sensitive words are detected, the chat disconnects. If sensitive words are not detected, the LLM produces a response that it runs through the same filtration process. The chat disconnects if the LLM’s response contains sensitive words. If not, the user sees a response to their query.

Even if a response has not been flagged by the algorithm’s filters, it is still monitored for “risky words” and the outputs are manually reviewed by in-house or contracted censors. The Qihoo 360 presentation says that the list of sensitive words is updated every 10 minutes and their list is shared with the government’s internet monitor.

Censoring specific topics is not new to Chinese tech, but as Will Knight at Wired points out, internet censorship could affect China’s chatbots in more subtle ways:

The algorithm trained on Chinese-language Wikipedia associated the words “democracy” closer to positive words such as “stability.” The algorithm trained on the censored Baike material represented “democracy” closer to “chaos,” more in line with the policy of China’s government.

Knight is referencing a 2021 study in which government policies influenced what the AI language algorithms (that will later be used in chatbots) considered positive or negative. For example, the Baike-trained algorithm gave positive scores to media headlines with the words “surveillance,” “social control,” and “CCP.” The Wikipedia-trained algorithm gave positive scores to headlines with the words “election,” “freedom,” and “democracy.”

In April the Cyberspace Administration of China provided draft rules requiring AI-produced content to “embody core socialist values,” referring to the 12 core socialist values outlined in 2012. The Chinese Communist Party is well aware that even if LLMs are trained on sanitized data, the output is still value-laden because word choice will always have certain connotations, something that even the smartest algorithm has trouble learning.