In a review of Shoshana Zuboff’s groundbreaking Surveillance Capitalism (2019), computer science historian Erik J. Larson recounts a 1950s conflict of ideas between two pioneers, Norbert Wiener (1894-1964) and John McCarthy (1927–2011). Wiener warned, in his largely forgotten book The Human Use of Human Beings (1950), about “new forms of control made possible by the development of advancing technologies.” McCarthy, by contrast, coined the term “artificial intelligence” (1956), implying his belief in “the official effort to program computers to exhibit human-like intelligence.” His “AI Rules” view came to be expressed not in a mere book but in — probably — hundreds of thousands of media articles warning about or celebrating the triumph of AI over humanity.

If you are skeptical of the basic message of those articles, you are not only right but in good company.

Larson, author of The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do (Harvard University Press, 2021) — one of the leading books of the year in AI* — sees Zuboff as a worthy successor to Wiener in opting for a more realistic view of what AI does in societies:

Harvard professor of business Shoshana Zuboff’s bestseller The Age of Surveillance Capitalism is a kind of reincarnation of Wiener’s focus on computation as control — and it couldn’t come at a better time, as worries about Big Tech, companies like Google, Facebook, Amazon, Microsoft and Apple, increasingly make news for a grab bag of broadly anti-human and anti-democratic violations…

Privacy issues as well as complaints about information monopolies and even mental health worries have all peppered the landscape of once lionized Silicon Valley companies who seemingly turned lead into gold, remaking our world with vast services and capabilities unthinkable even two decades ago. But Zuboff has performed perhaps the most valuable service of all, as her latest book takes the often disparate and inchoate feelings that Big Tech has run amuck and pulls them together into a framework that gets at the nub of the problem for us.

Erik J. Larson, “Wiener and Zuboff on Surveillance Capitalism” at Expensivity (May 20, 2021)

As Larson points out, originally, the ability of Big Tech to direct searches was a benefit. But the “digital exhaust” of information we leave behind (which it retains) has become a means of control:

The predictions translate into profits when tied to advertising clicks. Thus the age of the click-through rates emerged on the tail of the initially salutary motive to personalize online services. Click-throughs are big money, as Google and its investors soon discovered. And, subtly yet importantly, the AI algorithms powering Google turned toward the control paradigm that Wiener warned us might be the true legacy of the new science.

Erik J. Larson, “Wiener and Zuboff on Surveillance Capitalism” at Expensivity (May 20, 2021)

Larson contrasts Big Tech with China, saying “Silicon Valley doesn’t care about owning your soul or policing your thoughts” but, he confesses, it “sometimes seemingly does.” Indeed. Anyone following the attempted shutdown of social media competitor Parler (and many others, less publicized) would agree that it sometimes seemingly does look that way…

Although he mentions government hearings on the question, it’s unclear what government can do — or would even wish to do — about all this. Realistically, many politicians and their parties may feel more comfortable aligning with and benefiting from the largesse of Big Tech than with limiting its power. They may be very comfortable with a situation where, as Larson outlines, we do not know much about Big Tech or about the politicians but they know a great deal about us.

Larson goes on to note, citing Zuboff and recalling Wiener again,

If Google can get you to click on something, it hardly need concern itself with merely predicting your future behavior as a free-thinking person. Why? Because, in an important sense, you’ve ceded some of that freedom yourself.

Erik J. Larson, “Wiener and Zuboff on Surveillance Capitalism” at Expensivity (May 20, 2021)

Recently, Caitlin Bassett asked, here at Mind Matters News, whether the United States itself is beginning to behave like China. The results are not reassuring. For example, she shows, America has been flirting with policing by facial recognition for years. But a bigger problem is the extent to which we voluntarily let Big Tech know all about us and run our lives: “We may not live in a surveillance state under a Communist government, but by surrendering our data to Facebook, Google, Apple, and other internet giants, we have subjected ourselves to living within what Shoshana Zuboff calls ‘surveillance capitalism.’”

The difference has turned out to be that in China, the surveillance technology is in the hands of Big Government but in America, it is in the hands of Big Tech. Bassett quotes commentator Rod Dreher in Live Not By Lies: “Big Brother is not exactly who we expected him to be,” “…He’s a salesman, he’s a broker, he’s a gatherer of raw materials, and a manufacturer of desires. He is monitoring virtually every move you make to determine how to sell you more things, and in so doing, learning how to direct your behavior.” Netflix’s The Social Dilemma, has the story on that.

China is forcing the issue for us in a stark way. It has currently committed the resources of total surveillance to an all out war on religion. Obsessed with the fall of the Soviet Union, Party leaders are determined to use new laws and technologies to render religious influence impossible:

China scholars note that the CCP is very interested in studying how other authoritarian governments were toppled. The CCP, and Xi Jinping in particular, are obsessed with the fall of the Soviet Union. They don’t see the collapse as due to inherent weakness but rather due to tolerance of dissent. To them, it shows the importance of weeding out dissent and corruption as well as controlling the military from the top down. The military and other government elites must not have split loyalties, even religious loyalties. Thus they must be avowedly atheist and study Xi Jinping Thought. Even retired CCP officials and veterans are prohibited from practicing any religion.

The CCP also studied the Arab Spring protests (2010) which showed how social media can mobilize crowds. That ties into the CCP’s aggressive response to the Hong Kong democracy protests, which were mobilized online and the “strike hard” campaign against the Uyghurs after a series of violent conflicts between Han Chinese and Uyghurs (who are not ethnically Chinese).

The CCP also learned from the Arab Spring the importance of controlling the narrative and dividing the populace by blaming an ethnic or religious group.

Heather Zeiger, “Why the Chinese Communist party feels it must destroy religion” at Mind Matters News (May 23, 2021)

As Zeiger notes, persecution of religious groups, massively helped by intensive high tech surveillance, is not based on what the groups actually believe or teach but on whether their existence could, in principle, could pose a threat to the Communist Party.

Whether China succeeds or fails in wiping out religious belief may well predict the future of Big Tech control of humanity elsewhere.

• Note: As of this writing, May 24, 2021, Larson’s The Myth of Artificial Intelligence is #2 in Natural Language Processing and #4 in Computing Industry History.

You may also wish to read:

How Erik Larson hit on a method for deciding who is influential. The author of The Myth of Artificial Intelligence decided to apply an algorithm to Wikipedia — but it had to be very specific. Many measures of influence depend on rough measures like numbers of hits on pages. Larson realized that influence is subtler than that.

and

U.S. Postal Service secretly monitoring social media posts. Legal experts don’t understand why the Post Office is involved in online government surveillance. Analysts belonging to a United States Postal Service program called iCOP have been surveilling social media posts online for violent and inflammatory content. (Caitlin Bassett)