Mind Matters Natural and Artificial Intelligence News and Analysis

TagGary Smith

young-asian-business-team-people-meeting-in-office-stockpack-adobe-stock
young asian business team people meeting in office

The Death of Peer Review?

Science is built on useful research and thoroughly vetted peer review

Two years ago, I wrote about how peer review has become an example of Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure.” Once scientific accomplishments came to be gauged by the publication of peer-reviewed research papers, peer review ceased to be a good measure of scientific accomplishments. The situation has not improved. One consequence of the pressure to publish is the temptation researchers have to p-hack or HARK. P-hacking occurs when a researcher tortures the data in order to support a desired conclusion. For example, a researcher might look at subsets of the data, discard inconvenient data, or try different model specifications until the desired results are obtained and deemed statistically significant—and therefore Read More ›

close-up-of-calculation-table-printed-in-an-old-book-stockpack-adobe-stock
close up of calculation table, printed in an old book

An Illusion of Emergence, Part 2

A figure can tell a story but, intentionally or unintentionally, the story that is told may be fiction

I recently wrote about how graphs that use logarithms on the horizontal axis can create a misleading impression of the relationship between two variables. The specific example I used was the claim made in a recent paper (with 16 coauthors from Google, Stanford, UNC Chapel Hill, and DeepMind) that scaling up the number of parameters in large language models (LLMs) like ChatGPT can cause “emergence,” which they define as qualitative changes in abilities that are not present in smaller-scale models but are present in large-scale models; thus they cannot be predicted by simply extrapolating the performance improvements on smaller-scale models. They present several graphs similar to this one that seem to show emergence: However, their graphs have the logarithms of Read More ›

observing the data
In the System Control Room Technical Operator Stands and Monitors Various Activities Showing on Multiple Displays with Graphics. Administrator Monitors Work of  Artificial Intelligence.

Gary Smith’s New Book Reviewed in Washington Post

Smith argues that science itself is being undermined by the tools scientists use

Walter Bradley Center Senior Fellow Gary Smith’s book Distrust: Big Data, Data-Torturing, and the Assault on Science was reviewed in The Washington Post today. Smith is a frequent contributor to Mind Matters and teaches economics at Pomona College. In his new book, Smith argues that science itself is being “undermined” by the tools scientists use. Reviewer Abby Ohlheister writes, Smith, an economist whose work often examines the misuse of data and statistics in a variety of disciplines, argues that the current crisis of trust in science falls at the intersection of three forces: disinformation, data torturing and data mining. Disinformation, as Smith writes, is “as old as the human race,” but accelerated in speed and reach alongside social media. Data Read More ›

bulb brain
Creative Idea with Brain and Light Bulb Illustration, with Generative AI Technology

Dear Silicon Valley: You’re Over-Hyping ChatGPT

The abilities of these new chatbots are grossly overstated

Gary Smith and Jeffrey Funk, frequent Mind Matters contributors, co-wrote a piece at Salon on the over-exaggerated dreams big tech has for AI. They write, Silicon Valley’s pre-eminent leaders love prematurely predicting that their products will completely upend the world as we know it. The latest case study comes from Sam Altman, CEO of OpenAI, the company behind the ChatGPT AI chatbot that has gone viral for its convincing imitations of human writing. Two years ago, Altman wrote a manifesto, “Moore’s Law for Everything,” in which he forecast that artificial intelligence would make huge swaths of both white collar and blue collar jobs obsolete. -Smith & Funk, Don’t believe the hype: why ChatGPT is not the “holy grail” of AI research | Read More ›

hands-stockpack-adobe-stock
hands

Observing and Communing

What human art and literature do that AI can't

AI image generators like Midjourney or DALL-E are generally adept at capturing the accuracy of the human form. The concerns over copyright, job infringement, and general degradation of the visual arts via such AI are ongoing concerns for many artists and practitioners. However, a new New Yorker article by Kyle Chayka identifies a noticeable flaw in AI artwork: human hands. Missing the Big Picture Chayka begins by recalling an art class where he was asked to draw his own hand. It’s an assignment for beginners, and as behooves a novice, tempts the artist to focus more on the specific contours of the hand instead of the overall structure and form. The forest gets lost in the trees, so to speak. Read More ›

students-making-notes-stockpack-adobe-stock
Students making notes

Learning to Communicate

Why writing skills are so important, especially in today's artificial world

Educators have been shaken by fears that students will use ChatGTP and other large language models (LLMs) to answer questions and write essays. LLMs are indeed astonishing good at finding facts and generating coherent essays — although the alleged facts are sometimes false and the essays are sometimes tedious BS supported by fake references. I am more optimistic than most. I am hopeful that LLMs will be a catalyst for a widespread discussion of our educational goals. What might students learn in schools that will be useful long after they graduate? There are many worthy goals, but critical thinking and communication skills should be high on any list. I’ve written elsewhere about how critical thinking abilities are important for students Read More ›

artfcal

Let’s Take the “I” Out of AI

Large language models, though impressive, are not the solution. They may well be the catalyst for calamity.

When OpenAI’s text generator, ChatGPT, was released to the public this past November, the initial reaction was widespread astonishment. Marc Andreessen described it as, “Pure, absolute, indescribable magic.” Bill Gates said that the creation of ChatGPT was as important as the creation of the internet. Jensen Huang, Nvidia’s CEO, Jensen Huang, said that, “ChatGPT is one of the greatest things ever created in the computing industry.” Conversations with ChatGPT are, indeed, very much like conversations with a super-intelligent human. For many, it seems that the 70-year search for a computer program that could rival or surpass human intelligence has finally paid off. Perhaps we are close to the long-anticipated singularity where computers improve rapidly and autonomously, leaving humans far behind, Read More ›

little bot boi
Businessman holding a light chatbot hologram intelligence AI. Digital chatbot, chatGPT, robot application.Chat GPT chat with AI Artifice intelligent developers digital technology concept.

Does New A.I. Live Up to the Hype?

Experts are finding ChatGPT and other LLMs unimpressive, but investors aren't getting the memo

Original article was featured at Salon on February 21st, 2023. On November 30, 2022, OpenAI announced the public release of ChatGPT-3, a large language model (LLM) that can engage in astonishingly human-like conversations and answer an incredible variety of questions. Three weeks later, Google’s management — wary that they had been publicly eclipsed by a competitor in the artificial intelligence technology space — issued a “Code Red” to staff. Google’s core business is its search engine, which currently accounts for 84% of the global search market. Their search engine is so dominant that searching the internet is generically called “googling.” When a user poses a search request, Google’s search engine returns dozens of helpful links along with targeted advertisements based on its knowledge of the Read More ›

chat-bot-and-future-marketing-concept-chatbot-icon-hand-holding-mobile-phone-with-automatic-chatbot-message-screen-with-abstract-background-stockpack-adobe-stock
Chat bot and future marketing concept , Chatbot icon , Hand holding mobile phone with automatic chatbot message screen with abstract background

Let’s Call AI What It Really Is: Faux Intelligence

Gary Smith at Salon: While GPT-3 can string words together in convincing ways, it has no idea what the words mean

Pomona College business and investments prof Gary Smith warns Salon readers not to be too gullible about what human-sounding chatbots really amount to. He notes that in the 1960s, a pioneer chatbot called ELIZA convinced many psychiatric patients that they were interacting with a real psychiatrist. The machine simply repeated back their statements as questions, a popular psychiatric technique at the time because it generated more and more discussion — from the patient. The patients’ belief that they were interacting with a human being came to be called the Eliza effect. Has much changed? If you play around with GPT-3 (and I encourage you to do so) your initial response is likely to be astonishment — a full-blown Eliza effect. Read More ›

crypto-currency-background-with-various-of-shiny-silver-and-golden-physical-cryptocurrencies-symbol-coins-bitcoin-ethereum-litecoin-zcash-ripple-stockpack-adobe-stock.jpg
Crypto currency background with various of shiny silver and golden physical cryptocurrencies symbol coins, Bitcoin, Ethereum, Litecoin, zcash, ripple

Is Cryptocurrency Part of the Overall Future of Money?

Experts clash over whether crypto could replace government-issued currency

Recent discussions between WBC director Robert J. Marks and fellow engineers raise a question: Could new technology enable a global private currency to compete with government currencies? While the fact is not always evident in the Western world, government currencies depend on the stability of the government. Stories about people using discredited government banknotes to warm themselves, etc., are not fiction. And today what about the unbanked billions of the world who work and create value but do not have access to financial institutions? So in, for example, a national emergency, would cryptocurrency be any help? The option of private electronic currency is comparatively new and, as we might expect, expert views differ. We present them for your reflection (not Read More ›

scientists-testing-in-lab-stockpack-adobe-stock
Scientists testing in lab.

Will AI Really Change Drug Development? Not So Fast…

Jeffrey Funk and Gary N. Smith note that AI was not significant in the development of COVID vaccines. Financial incentives ruled

Something to know before you invest or entertain high hopes: Jeffrey Funk and Gary Smith published a recent article in Salon that offers a free cold shower. Some realities they cite: Most of the expense of drug development is in clinical trials on human beings, which can’t be automated. Any attempt to save time or money would come at identifiable costs in accuracy. Yes, COVID vaccines were a banner achievement for speedy drug development. But AI played little or no standout role in the process: Determined to get a COVID-19 vaccine to the public before the November 3, 2020, presidential election, the U.S. government devoted $14 billion to support the pharmaceutical companies’ vaccine efforts. The government agreed to pay Pfizer Read More ›

unicorn-on-gold-cube-mountain-start-up-illustration-concept-of-leader-on-a-market3d-rendering-3d-illustration-stockpack-adobe-stock
unicorn on gold cube mountain . start up, illustration concept of leader on a market.3d rendering. 3d illustration.

Warning: Unicorn Stocks May Be Nearing Bankruptcy, Fire Sales

Uber, Airbnb, and DoorDash seem so lifestyle-friendly … they even became part of our vocabulary

In an article today, Jeffrey Funk and Gary Smith puzzle over the way investors have “rained cash” on unprofitable companies like Uber, Airbnb, DoorDash, etc. Yes, they are icons of popular culture. No, unlike Apple and Google, they do not make money: Here are their stats from the table the authors offer: Company Founded Funds Raised Cumulative LossesUber Technologies 2009 $25.2 billion $31.7 billionAirbnb 2008 $6.0 billion $6.0 billionDoor Dash 2013 $2.5 billion $4.6 billion Only one of the 15 companies they list — including some other culture icons — has ever had a profitable quarter: Any hopeful arguments that profitability is just around the corner ring hollow when every company is at least nine years old and two are Read More ›

robotic vacuum hardwood
robotic vacuum cleaner on laminate wood floor smart cleaning technology

Turns Out, Computers Are Not Vacuuming Up All Our Jobs

Far from it, we can hardly find all the people we need to manage the computers

Let’s start with radiologists: In 2016 Turing Award Winner Geoffrey Hinton advised that “We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists.” Six years later, the number of radiologists has gone up, not down. Researchers have spent billions of dollars working on thousands of radiology image-recognition algorithms that are not as good as human radiologists. Jeffrey Funk and Gary Smith, “The right and wrong way to use artificial intelligence” at New York Daily News (August 6, 2022) Technology researcher Jeffrey Funk and business prof Gary Smith could — and probably will — fill a book with examples, some of which they list and link to at Read More ›

close-up-of-stock-market-trader-looking-at-graph-stockpack-adobe-stock
Close up of stock market trader looking at graph

Not All False Prophets Are Promoting Religions…

False finance prophets’ credentials or charisma are much more impressive than their track records — but we need a way to tell

Recently, our authors Jeffrey Lee Funk and Gary Smith alerted MarketWatch readers to the problem of “false prophets” in the investment world: We could write a long book about false prophets on Wall Street. What is interesting is how easily people are enchanted by charismatic personalities — some who peddle advice, some who run companies. A decade ago, for example, Yahoo tried to save itself by paying almost $1 billion to five charismatic CEOs (Terry Semel, Jerry Yang, Carol Bartz, Scott Thompson, and Marissa Mayer), four of them outsiders, who were hired over a five-year period and arguably did more harm than good. Jeffrey Lee Funk and Gary N. Smith, “Sensible stock investors put their money on a company’s real Read More ›

Go-game

Would AI Still Win at Go If the Board Shrunk: 19 to 17 Spaces?

No, say Jeffrey Funk and Gary Smith — and would-be investors need to grasp AI’s weaknesses as well as strengths, for success

Statistician Jeffrey Lee Funk and business prof Gary N. Smith offer a warning for investors: Some AI stocks have been good investments but most high tech unicorns never pay off. It’s a not surprising, they say, when we consider that AI is powerful but brittle. An example they offer: AI easily beats humans at the game of go which features a 19 × 19-square board. If the game switched to a 17 × 17-square board, humans would quickly adjust but AI would flounder. They offer examples of how this sort of limitation plays out in the real world, including the true tale of a hapless AI-driven insurance company: An insurance company with the quirky name Lemonade was founded in 2015 Read More ›

engineers-meeting-in-robotic-research-laboratory-engineers-scientists-and-developers-gathered-around-illuminated-conference-table-talking-using-tablet-and-analysing-design-of-industrial-robot-arm-stockpack-adobe-stock
Engineers Meeting in Robotic Research Laboratory: Engineers, Scientists and Developers Gathered Around Illuminated Conference Table, Talking, Using Tablet and Analysing Design of Industrial Robot Arm

At Salon, Funk and Smith Take On “Stealth AI Research”

All we know for sure about the claims about Google AI’s LaMDA showing human-like understanding is that, since 2020, three researchers who expressed doubt/concerns were fired

Yesterday at Salon, Jeffrey Funk and Gary N. Smith took a critical look at “stealth research” in artificial intelligence. Stealth research? They explain, A lot of stealth research today involves artificial intelligence (AI), which Sundar Pichai, Alphabet’s CEO, has compared to mankind’s harnessing of fire and electricity — a comparison that itself attests to overhyped atmosphere that surrounds AI research. For many companies, press releases are more important than peer review. Blaise Agüera y Arcas, the head of Google’s AI group in Seattle, recently reported that LaMDA, Google’s state-of-the-art large language model (LLM), generated this text, which is remarkably similar to human conversation: Blaise: How do you know if a thing loves you back? LaMDA: There isn’t an easy answer Read More ›

transition-effect-in-bar-chart-statistics-and-bright-windows-stockpack-adobe-stock
Transition effect in bar chart statistics and bright windows

Studies Based On Data Mining Can Turn Out To Be Nonsense

As business prof Gary Smith explains at Bloomberg, we can find a great deal of nonsense if all we rely on is a search engine

Many of the studies represented to us in media are not nearly as reliable as we would like to believe, as Gary Smith explains at Bloomberg. He starts with statistician John Ioannidis pointing out that of 34 “highly respected” medical studies and found that only 20 were confirmed by the Reproducibility Project.: I wrote a satirical paper that was intended to demonstrate the folly of data mining. I looked at Donald Trump’s voluminous tweets and found statistically significant correlations between: Trump tweeting the word “president” and the S&P 500 index two days later; Trump tweeting the word “ever” and the temperature in Moscow four days later; Trump tweeting the word “more” and the price of tea in China four days Read More ›

scoring basket
scoring during a basketball game - ball in hoop

Luck Matters More Than Skill When You’re at the Top

What? Shouldn’t it be the other way around? No, because… Prof. Gary Smith explains

With basketball fever at a high pitch… when LA Times sportswriter Jim Alexander talked to Pomona College business prof Gary Smith about what it takes to win, he got a different answer than some might have expected. If you are really good, it takes luck to win, Smith explained. What? Shouldn’t it be the other way around? No, because… “You can take the four best golfers in the world – any sport, but let’s do golf because it’s head-to-head,” Smith said in a phone conversation this week. “And they play a round of golf and see who gets the lowest score, and it’s pretty much random. Nobody’s going to win every single time. One guy might win more than 25 Read More ›

concept-creative-idea-and-innovation-hand-picked-wooden-cube-block-with-head-human-symbol-and-light-bulb-icon-stockpack-adobe-stock
Concept creative idea and innovation. Hand picked wooden cube block with head human symbol and light bulb icon

Computer Prof: We Can’t Give Machines Understanding of the World

Not now, anyway. Melanie Mitchell of the Santa Fe Institute finds that ever larger computers are learning to sound more sophisticated but have no intrinsic knowledge

Last December, computer science prof Melanie Mitchell, author of Artificial Intelligence: A Guide for Thinking Humans (2019), let us in on a little-publicized fact: Despite the greatly increased capacity of the vast new neural networks. they are not closer to actually understanding what they read: The crux of the problem, in my view, is that understanding language requires understanding the world, and a machine exposed only to language cannot gain such an understanding. Consider what it means to understand “The sports car passed the mail truck because it was going slower.” You need to know what sports cars and mail trucks are, that cars can “pass” one another, and, at an even more basic level, that vehicles are objects that Read More ›

san-francisco-aerial-view-from-sea-side-port-of-san-francisco-in-the-front-city-downtown-and-skyscrapers-at-sunrise-stockpack-adobe-stock
San Francisco aerial view from sea side. Port of San Francisco in the front. City downtown and skyscrapers at sunrise.

When Silicon Valley Turns From Hype Over Vaporware to Fraud…

Jeffrey Funk and Gary Smith discuss the famous Theranos case, which resulted huge losses and in convictions for fraud

In a column published today at MarketWatch, Jeffrey Funk and Gary Smith talk about that unpleasant subject, the shady side of Silicon Valley. They’re not looking at the unicorns naively chasing rainbows but rather the cases of apparently deliberate deception. One of them is vaporware— announcing a product that won’t really exist any time soon (perhaps in the hope of dissuading potential buyers from investing in a competitor’s product). Another is “fake it until you make it,” the topic of today’s column. Investors sign on by throwing money at the company, which the company then spends trying to develop what it said it already has. Either way, the company keeps lying as long as necessary, or until its cover is Read More ›