One of the most concerning aspects of social media is that much of its influence evades our notice. We don’t realize that we’re being influenced, or shaped to think a certain way, or view the world through a specific lens. Online algorithms reinforce our preferences but can also subtly introduce us to content we’d never choose on our own in a million years. This probably explains why I’m tempted to watch videos of people getting “scare pranked” on YouTube on an almost daily basis. I would never intentionally seek out those humorous videos, but for some reason or another, embedded in the matrix, that sort of content started popping up on my screen.

Large Language Models (LLMs) like ChatGPT are more “conversational” technologies, directly addressing users, so their persuasive power is more overt. What’s more, tech leaders like OpenAI’s Sam Altman are celebrating AI’s ability to do this. A new report from Wired illustrates how AI could be a powerful tool used to shape people’s outlook. Of course, Altman considers the positive potential for this. AI could encourage people to exercise better, or get on an effective diet, or give an overview of the strengths and weakness of politicians. Thrive, a health company, is introducing “Thrive AI” to try and help people get healthier. Arianna Huffington, the company’s CEO, wants the AI to be a “personal coach.” OpenAI is helping fund the project, and Altman is all for it. Will Knight writes,

Altman and Huffington write that Thrive AI is working toward “a fully integrated personal AI coach that offers real-time nudges and recommendations unique to you that allows you to take action on your daily behaviors to improve your health.”

Their vision puts a positive spin on what may well prove to be one of AI’s sharpest double-edges. AI models are already adept at persuading people, and we don’t know how much more powerful they could become as they advance and gain access to more personal data.

-Will Knight, OpenAI Is Testing Its Powers of Persuasion | WIRED

As AI potentially gains access to more user data, it could get even better at “learning” our preferences, desires, habits, and goals. If social media’s overreach has served us any indication, AI could get scarily good at persuading people to take certain action, hold a belief, or feel a certain way. Knight goes on in the article to talk about chatbot girlfriends. The relational appeal of talking to a bot is important to note, too. If it feels like we are engaging with a real person, then sharing more private and intimate information may get all the easier to do. The question is: who’s reading?