Andy Crouch is a man who wants to evade the algorithm, so he doesn’t post much on X, but when he does, it’s generally an insightful nugget meant to be noted and digested. Crouch, a tech critic and a partner at Praxis Labs, doesn’t think AI will do all the wild stuff some of the futurists claim, but he does believe a belief in AI’s potential outlandish capacities, like consciousness, could be harmful. He posted on X,

The analogy with magic is one Crouch has employed before in his extrapolations of technology. His thoughts echo economist Gary Smith’s frequent musings on this subject. The real danger of AI is not that it will become smarter than humans, but that we think that it already has. So, we depend on it to accomplish complex tasks or to fulfill certain roles that, by design, it simply can’t appropriate. That’s why conversations on AI must be tempered by a respect for the uniqueness of the human person. Crouch understands that, and we need more voices that do, too. Smith writes with Jeffrey Funk,

We are deeply skeptical of the ability of LLMs to generate novel ways to boost productivity. They are much more likely to regurgitate words that others have already written, spiced up with misleading hallucinations. No matter how many words they train on and no matter how many human trainers correct their mistakes, they will continue to lack the critical thinking skills required to evaluate new ideas.

Smith & Funk, Universities Should Prioritize Critical Thinking Over Large Language Models | Mind Matters

AI’s magical hype will continue to hypnotize, but its fundamental limits are stark, and need to be acknowledged.