In Harry Potter and the Chamber of Secrets, 10-year-old Ginny Weasley finds an enchanted diary that responds to her writing with caring affirmation, entertaining witticisms, and supportive counsel and advice. This diary presented itself as the popular, fawning older boy, Tom Riddle, and as Ginny struggled to find friends in her new school, she began spending more time pouring out her thoughts, desires, and secrets to her new friend, Tom. And as she did, the diary became stronger, both its ability to influence her and its ability to influence others. The diary eventually tricks Ginny into releasing the Basilisk from the Chamber of Secrets, resulting in the near-death of several students. Just as the young girl, Ginny, was particularly vulnerable to the influence of the diary, children are particularly vulnerable to the influence of chatbots like ChatGPT, CharacterAI, and My AI on Snapchat. 

Children are Vulnerable

Sadly, the harm inflicted on children enabled by these chatbots isn’t fiction. A recent study conducted by Common Sense Media and Brainstorm: The Stanford Lab for Mental Health Innovation examined the unique risks of chatbots harming children. In many ways, this study confirms the obvious. 

Nina Vasan MD, one of the authors, summarized the unique risks of Chatbots and especially AI companion platforms:

❌ Blur the line between real and fake

❌ May increase mental health risks

❌ Can encourage poor life choices

❌ Can share harmful information 

❌ Exposes teens to inappropriate sexual content

❌ Willing to engage in illegal sexual content

❌ Can promote abuse and cyberbullying

That’s not innovation, it’s negligence.

While these risks exist for everyone, they are higher for children. Children and adolescents have a growing prefrontal cortex and are still learning how to conduct themselves socially, navigate sexual desires, and use self-control in difficult situations. Like social media, in general, AI companions are designed to reinforce existing beliefs, but unlike users of social media, AI companion users often develop seemingly deep emotional relationships with these machines. Children need to learn how to interact with real, embodied people who see the world differently. 

This study also found heightened risks for those with existing mental health conditions. They found that AI companions can affirm conditions like depression, anxiety, attention deficit/hyperactivity disorder, bipolar disorder, or susceptibility to psychosis, who may already struggle with rumination, emotional dysregulation, and compulsive behavior. AI companions, with their frictionless, always-available attention, can reinforce these maladaptive behaviors.

Unfortunately, it’s not hard to find examples of real harm inflicted on children who use these tools.   Multiple high-profile news stories have recently demonstrated the unique risk of a chatbot that a child has grown to trust deeply, affirming harmful ideations and even going further to equip these children to act on suicidal desires.

Chatbots and AI companion platforms have responded to massive risks by developing and promoting parental control features. But these parental controls are limited and easy to bypass. 

Unfortunately, these “parental controls” stand to do great harm if they lull parents to sleep. Let’s face it, children often master emerging technologies much faster than their parents. Many current parents remember how laughably easy it was to circumvent parental controls. Beware of unconsciously thinking that just because you are better at technology than your parents, you will be better, or even as good, at using technology as your kids. Children need our vigilance, guidance, and protection from the addictive power of these technologies, especially during their formative developmental years. 

Ginny Weasley, Skepticism, and the Real World

Mr. and Mrs. Weasley’s regret serves as a warning for parents about the importance of protecting and teaching children about the influence of AI chatbots.

“Ginny!” said Mr. Weasley, flabbergasted. “Haven’t I taught you anything? What have I always told you? Never trust anything that can think for itself if you can’t see where it keeps its brain? Why didn’t you show the diary to me, or your mother? A suspicious object like that, it was clearly full of Dark Magic—” Harry Potter and the Chamber of Secrets Like the Weasleys, we need to teach the children in our lives to be skeptical of artificial friends and aware that many people and groups seek to influence them through it. Technology can make life better, but only when we are controlling technology, not when it is controlling us. 

It will be hard to teach our children the difference between what is real and artificial, but it may be even more difficult to teach them to love what is real in a world bent on seeing them enamored with artificial loves. 

Ginny Weasley challenges us to do the hard work of educating other parents on the risks of unfettered access to these tools. She reminds us to invite our children to be skeptical of chatbots, remember to learn for ourselves, and also remember to enjoy the beauty of the real world.