The debate about AI continues to diversify and intensify. Amidst the disagreements between techno-optimists and doomsayers, there is also a debate about the extent to which AI is currently used and will be used in the future.  McKinsey estimates that “generative AI could add the equivalent of $2.6 trillion to $4.4 trillion annually across the 63 use cases [it] analyzed,” five years after it said AI in general can deliver $13 trillion in economic value by 2030. Those are big numbers.

On the other hand, Gary Marcus has questioned the existence of successful use cases, consistent with various surveys that show a slow uptake of generative AI. In addition, Gary Smith has repeatedly shown the limitations of generative AI in various situations. Another sign of slow adoption is that many companies have banned them (or are considering banning them) because of problems from security to hallucinations. These are problems that will require much work.

This article focuses on one of these debates: how to define successful use cases. The first issue is that most analyses emphasize tasks, such as e-mail and reports, and not processes, even though processes are much more important. Processes integrate the output of tasks from different employees into something the end customer wants and define the way in which task quality is defined. Tasks such as e-mail and reports only provide value when they contribute toward value for the end customers, a difficult result to achieve.

A second issue is that many companies will not quickly adopt AI for a specific application if it requires a lot of customization. Consider three widely discussed use cases: research assistants, chatbots, and hiring. One reason why these proposed use cases aren’t being widely used yet is that they can’t be done by a single product, they require lots of customization, and thus aren’t easily implemented.

What do I mean? Word processing and spreadsheets were killer apps for the first personal computers (and later power point) because users merely had to purchase a computer and the software to use them. Although there were competing versions of software (and hardware), each version of the general-purpose software would enable users to complete documents without customization of the software by companies or users. Of course, later companies introduced much more sophisticated systems for handling not just documents but internal processes such as supply chain management and customer relationship management.

Similarly, e-commerce and news content were killer apps for the Internet because single products could again satisfy many customers. Amazon.com is just one example of a company that enabled consumers and businesses to purchase products without visiting stores, and often for lower prices. The reduction in inventory and store space are big reasons for the lower costs. Similar arguments can be made for the simplicity of accessing news and other content.

It is hard to find these types of use cases in AI, even with generative AI, the type of AI that took the world by storm in 2023. The fact is that research assistant, chatbot, and hiring applications can’t be done by a single external product across multiple industries. One reason they don’t work for multiple industries is because each industry has different types of knowledge, products, and skills, and thus the product must be trained for each industry.

For instance, a scientist working in the biotech industry will require a different type of assistant than an engineer working in the automobile industry. The same holds for other industries because different industries use different words in their work, which reflects the widely different knowledge, products, and skills. These different words may mean different things in different industries, and unless the tools understand these differences, they will be much more likely to hallucinate and get things completely wrong.

A different assistant (and thus training) may even be required for each company not only because different words are used, but for security reasons. Thus, many companies must develop their own tools, and this will take time. One reason it will take time is because these tools can’t be allowed to leak internal data, something that still occurs for custom chatbots.

The same logic holds for chatbots that handle customer inquiries and for systems used to hire people. Different words will be relevant for different industries, and perhaps for different companies, and thus a lot of customization, including training of the system on different data, will be required.

Some readers will say “no.” They will claim that research assistants will be as easy to implement as the applications of word processing, spread sheets, and power point that I mentioned earlier. In fact, some of these tools are even incorporating generative AI, so proponents argue that it is already happening.

In addition to the problems that I cited above, productivity improvements from these tools have become harder to obtain as diminishing returns have emerged. Word processing was a general tool, one that was introduced in an age of few documents, quantitative analyses, and presentations. Today, however, we are awash in those things and thus we need tools that help us do these things better, not just faster.

We need better ideas for scientific papers, new businesses, and new technologies. Evidence that existing ideas aren’t so good can be seen in the big startup losses, slow diffusion of technology, and slow rate of productivity improvement.

We Need Better, Not Just Faster

Thus, we need much better research assistants, chatbots, and hiring software than generative AI suppliers are currently offering. We need not only less hallucinations; we need more novelty, and more insights into what is wrong and can be done better. Current generative AI systems are designed to give us the most common solutions, and not the new ones we need. Their answers merely reflect what has been uploaded to the web and not the new ideas that are needed. Disbelievers should read Gary Smith’s many tests of generative AI in Mind Matters, of which here is a recent test of the Monty Hall problem.

There are some industries for which I am more optimistic than others. For instance, as I wrote in a previous article,

“The most successful cases may involve Hollywood’s creation of TV dramas and movies because the output is images and words, and AI helps companies manage the processes that produce this output. Not only are some commercials and short videos being made with generative AI, generative AI can help manage processes such as storyboarding, set design,  and visual effects, along with the overall process of fitting together multiple scenes that are recorded separately but then require fillers that can be done with generative AI.”

We need proponents of AI to better articulate the challenges of implementing AI, particularly the leaders of Silicon Valley, such as Chris Anderson and Marc Andreessen, who have consistently over-hyped AI and other technologies over the last few years. Similarly, Google and other companies have repeatedly misrepresented their product’s capabilities in videos and statements, and these types of actions will only slow AI’s implementation as potential users spend endless resources trying to implement something that is much harder than proponents have claimed. Recent admissions by OpenAI’s Chief Operating Officer are a good step in the right direction.