Mind Matters Natural and Artificial Intelligence News and Analysis

CategoryPeer Review

portrait-of-a-beautiful-lion-lion-in-the-dark-stockpack-adobe-stock
Portrait of a Beautiful lion, lion in the dark

Will the Real “Predatory Journal” Please Stand Up?

Large publishers serve themselves by painting "predatory journals" with a broad brush

The scientific publishing industry has been on a hunt for what it calls “predatory journals.” They want to make sure that all scientific publications occur in “legitimate” and “reputable” journals. Additionally, they encourage scholars to avoid “predatory” journals which are there merely to enrich themselves by having you pay for access. While I agree with these ideas in principle, I’ve noticed more and more that the way that these principles are applied has been, well, incredibly self-serving for the journals. To begin with, let’s look at a commentary on predatory journals published in 2019 in the journal Nature: Predatory journals are a global threat. They accept articles for publication — along with authors’ fees — without performing promised quality checks for issues such…

many-assorted-old-books-lays-in-the-stacks-as-background-front-view-close-up-stockpack-adobe-stock
Many assorted old books lays in the stacks as background front view close up

Will the Sokal Hoaxes Worsen the Academic Echo Chamber?

When only mainstream thinking is allowed, insularity and echo chambers are the result

The latest spate of academic hoaxes, which includes not only the latest Sokal hoax, but also fake papers being published in journals such as the Arabian Journal of Geosciences, is a major cause for concern. While the hoaxes themselves are problematic, what is likely to be worse is the ongoing fallout for researchers and new ideas. Academics has long had a problem of being insular. Many papers are published on the basis of your status in academics, not the quality of the paper itself. This is not to say that the papers didn’t legitimately pass peer review (though that is questionable in some cases), but rather that the editor decides which papers are “worthy” of peer review based on the editor’s knowledge…

bulls-eyes-and-head-close-up-black-and-white-photo-of-a-bull-stockpack-adobe-stock
Bull's eyes and head close-up, black and white photo of a bull.

Be On the Lookout for More Sokal Hoaxes

If you spot a Sokal hoax, let us know by tagging @cnaintelligence on Twitter

In a previous article, we noted that a group of researchers have been testing the rigor of social science journals by submitting fake articles and data in order to demonstrate problems in these branches of academics. These hoaxes, known as Sokal hoaxes (named after the original hoaxer, Alan Sokal), have now begun their third round, with the first detected paper in the journal Higher Education Quarterly. The paper, “Donor money and the academy: Perceptions of undue donor pressure in political science, economics, and philosophy,” has now been retracted, but it looks like this is not quite the end of the matter. The Chronicle of Higher Education managed to get in contact with the Sokal hoaxers. While their identities are anonymous, they responded to emails sent to the…

black bull
Portrait of bull in profile

Are Sokal Hoaxes Really Helping Reform Science?

The evidence is mixed. The current prank on Higher Education Quarterly prompts some questions
It happened. Again. In 1996, physicist Alan Sokal submitted a mostly-nonsense paper to the journal Social Text. The purpose? To show that the social sciences, especially the critical theorists, were not intellectually rigorous. His paper, “Transgressing the Boundaries: Towards a Transformative Hermeneutics of Quantum Gravity,” was written so that anyone who was familiar with quantum gravity would know that it was a joke. Therefore, when it was accepted, it was obvious that no rigorous review process was in place. The journal apparently just accepted anything that sounded “intellectual-ish” and conformed to the editors’ idea of what scholarship looked like. It became known as the Sokal hoax. In 2017, a group of researchers began a similar hoax, later tagged as Sokal Squared. The goal, again, was to show that what the authors call “grievance studies” are undermining real scholarship by allowing nonsense to pass as scholarship. They managed to get several of their fake papers published, for example, Human reactions to rape culture and queer performativity at urban dog parks in Portland, Oregon (May 22, 2018). In fact, for some of the papers that hadn’t been published by the time the experiment ended, the reviewers requested more outlandishness. That brings us to the present day. The education journal Higher Education Quarterly,” which is run by Wiley (a major academic publisher), published a new paper titled “Donor money and the academy: Perceptions of undue donor pressure in political science, economics, and philosophy.” If you take the alleged names of its authors, who cannot be traced to any institution of higher education, “Sage Owens” and “Kal Avers-Lynde III,” the initials spell out: S.O.K.A.L. III It is unclear at this writing what this third iteration of the Sokal Hoaxes is trying to prove. At Power Line, Steven Hayward suggests that the overall goal is to show that any paper can be published if it stokes the biases of leftist editors. If so, the hoaxers have pushed the right buttons: the Koch Brothers, the Federalist Society, right-wing interference… The problem? As search has demonstrated, the funders they study don’t actually fund the research they are studying and some of the organizations have been defunct through the entire period of study. One of them appears entirely fictitious. While we still don’t know who wrote “SOKAL III” or why, I think it is good to take a moment and reflect on academic hoaxes in general, and what they indicate. The goal of the hoaxers is to demonstrate that critical theory and “grievance studies” journals are only pseudo-academic. They want to show that obviously wrong papers get through as long as they conform to the style and expectations of the journal. So did these hoaxers succeed in their aim? To begin with, I should say that it is not the job of an editor or reviewer to catch falsified data. Journals assume that data associated with an article is legitimate and was legitimately obtained. Now, there are times when data is obviously wrong, and a worthwhile editor or reviewer should catch that. However, it is not the job of the reviewers to verify or repeat the experiment. They trust the reporting and merely make sure that the reasoning about the data is correct. Therefore, a proper hoax must illustrate more than just acceptance of falsified data. It has to illustrate acceptance of wrong reasoning about that data. Or, at minimum, the data must be so obviously wrong that any worthwhile reviewer would be able to spot it. However, there are other requirements of a proper experiment that the “Sokal squared” hoaxers seem to have left out. In order to show that the social science journals they targeted are more problematic than science journals, they would have needed a control group. Note that none of the Sokal hoaxers tried submitting nonsense to a science journal. Even other Darwinists often complain about “just-so” stories in evolutionary biology, and especially in evolutionary psychology. So why did the hoaxers not dream up an equivalently outrageous just-so biology story to see if it would pass peer review? Were they afraid to know the results? History says that, as long as you are proposing an evolutionary “just so” story, there is almost no idea that is too absurd to be published in even the topmost journals. The original Sokal hoax, too, had problems. The journal it was submitted to, Social Text, was not a peer-reviewed journal. In other words, the journal was not even claiming to have the same scholarship standards as science journals. In fact, they were relying on Sokal himself to provide the expertise on physics, which was the primary reason for including him. So, while I am in basic agreement with the hoaxers about the general quality of critical theory and “grievance studies” journals, I find it odd that the hoaxers, so certain of their proposition, seem to be unscientific in their approach to finding out the answers. In fact, we don’t even need to go into hoaxes to see that the same sort of implicit bias can infect ordinary science journals. As noted by Michael Crichton and followed up on here at Mind Matters News, science journals have been ready to publish whatever the latest trendy topic is for decades, no matter how bad the science content. Then, for topics that are less trendy, they will sometimes retract legitimate papers which have been properly reviewed simply because the topic is out of favor. If an idea is in favor, then, as we’ve seen, the requirements for rigor are relaxed significantly. So, while I appreciate the attempts of the Sokal hoaxers to demonstrate biases and poor scholarship in the social sciences, I fear that those in the harder sciences will use this as an excuse to ignore their own problems with bias and poor scholarship. Note: “SOKAL III” has now been retracted. You may also wish to read: Twenty years on, aliens still “cause global warming” Over the years, the Jurassic Park creator observed, science has drifted from its foundation as… Read More ›
father-and-son-competing-in-video-games-at-home-stockpack-adobe-stock
Father And Son Competing In Video Games At Home

Why Did Video Gamers Uncover Fraud More Easily Than Scientists?

Video gamers are subject, a psychologist tells us, to much more rigorous constraints than scientists

In a recent article at The Atlantic, King’s College psychologist Stuart Ritchie, author of Science Fictions: How Fraud, Bias, Negligence and Hype Undermine the Search for Truth (2020), has noted a curious fact: Video gamers are much quicker to spot fraud than scientists. The video game fraud he focuses on involved a gamer’s claim that he had finished a round of Minecraft in a little over 19 minutes, a feat he attributed, Ritchie tells us, to “an incredible stretch of good luck.” “Incredible” is the right choice of word here. “Dream,” as the player was known, later admitted — in the face of skepticism — that he had “inadvertently” left some software running that improved his game — thus disqualifying…

concept-of-different-emotions-drawn-on-colorfull-cubes-wooden-background-stockpack-adobe-stock
concept of Different emotions drawn on colorfull cubes, wooden background.

Emotion Recognition Software Use Spreads While Science Is Doubted

Emotional recognition software has been coming under fire for misuse and racial bias for some time

An editor at AI Trends notes The global emotion detection and recognition market is projected to grow to $37.1 billion by 2026, up from an estimated $19.5 billion in 2020, according to a recent report from MarketsandMarkets. North America is home to the largest market. John P. Desmond, “Market for Emotion Recognition Projected to Grow as Some Question Science” at AI Trends (June 24, 2021) But the software has been coming under fire for misuse and racial bias for some time: “How people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation,” stated the report, from a team of researchers led by Lisa Feldman Barrett, of Northeastern University,…

stack-of-papers-isolated-on-white-background-stockpack-adobe-stock
Stack of papers isolated on white background

A Vulnerable System: Fake Papers and Imaginary Scientists

Publication counts and citation indexes are too noisy and too easily manipulated to be reliable

In the last two posts, we examined how scientific publication has ceased to be a good measure of scientific accomplishment, and how the peer review system is being gamed by unscrupulous publishers and researchers alike. Now, we will continue the discussion on the undermining of scientific publication using two examples: SCIgen and citation counts. SCIgen In 2005 three MIT graduate computer science students created a prank program they called SCIgen for using randomly selected words to generate bogus computer-science papers complete with realistic graphs of random numbers. Their goal was “maximum amusement rather than coherence,” and also to demonstrate that some academic conferences will accept almost anything. They submitted a hoax paper titled, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,”…

lying-businessman-holding-fingers-crossed-behind-his-back-stockpack-adobe-stock
Lying businessman holding fingers crossed behind his back

Gaming the System: The Flaws in Peer Review

Peer review is well-intentioned, but flawed in many ways

Last time, we examined how scientific publication has ceased to be a good measure of scientific accomplishment because it has now become a target, following Goodhart’s Law. In today’s post, we will continue that examination by turning to the peer review system, and how that system is being gamed by unscrupulous publishers and researchers alike. In theory, the peer review process is intended to ensure that research papers do not get published unless impartial experts in the field deem them worthy of publication. Peer review is well-intentioned, but flawed in many ways. First, the best researchers are incredibly busy and naturally more inclined to do their own research than to review someone else’s work. Thus, peer review is often cursory or…

custom-library-stockpack-adobe-stock
Custom library

Publish or Perish — Another Example of Goodhart’s Law

In becoming a target, publication has ceased to be a good measure

The linchpin of scientific advances is that scientists publish their findings so that others can learn from them and expand on their insights. This is why some books are rightly considered among the most influential mathematical and scientific books of all time:  Elements, Euclid, c. 300 B.C.Physics, Aristotle, c. 330 B.C.On the Revolutions of Heavenly Spheres, Nicolaus Copernicus, 1543Dialogue Concerning the Two Chief World Systems, Galileo Galilei, 1632Mathematical Principles of Natural Philosophy, Isaac Newton, 1687The Origin of Species, Charles Darwin, 1859 As Newton said, “If I have seen further it is by standing on the shoulders of Giants.” It seems logical to gauge the importance of modern-day researchers by how much they have published and how often their research has been cited…

scientific-publication-stockpack-adobe-stock.jpg
scientific publication

Inside the Economics of Science Papers

Here’s an inside look at who pays if you read for free

When a scholarly paper is published, someone has to pay. Publishers like Institute of Electrical and Electronic Engineers (IEEE), my professional society, and Springer charge big bucks to read their papers. The fees are billed to individual subscribers and, more commonly, to companies and universities who want to give their employees access to the papers. My own university, Baylor, like most research universities, has a considerable library budget on account of these publisher fees. There is growing pressure to kill these fees in favor of “open access” to scholarly papers. Thus, anyone can read a scholarly paper at any time for free. Free access takes the money from the pockets of publishers so they push back. Someone has to pay,…

castaway-in-bureaucracy-stockpack-adobe-stock.jpg
Castaway in bureaucracy

Gregory Chaitin on How Bureaucracy Chokes Science Today

He complains, They’re managing to make it impossible for anybody to do any real research. You have to say in advance what you’re going to accomplish. You have to have milestones, reports

In last week’s podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin on how Stephen Wolfram’s software has taken much of the drudgery out of math. At the same time, in Chaitin’s view, a threat looms: A new, more bureaucratic, mindset threatens to take the creativity out of science, technology, and math: https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 19:45 min. A partial transcript, Show Notes, and Additional Resources follow. Robert J. Marks: I was sitting down tallying, I think, the intellectual giants that have introduced new mathematical ideas, brand new. I was thinking of people like Claude Shannon, Lotfi Zadeh, yourself… I don’t know if we…

black-mathematics-board-with-formulas-stockpack-adobe-stock.jpg
black mathematics board with formulas

Why Don’t We See Many Great Books on Math Any More?

Decades ago, Gregory Chaitin reminds us, mathematicians were not forced by the rules of the academic establishment to keep producing papers, so they could write key books.

In our most recent podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin on many things mathematical, including whether math is invented or discovered. This time out, Chaitin talks about why he thinks great books on math, advancing new theorems, aren’t written much any more: https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 02:49 min. A partial transcript, Show Notes, and Additional Resources follow. Robert J. Marks: You don’t hear the word “scholarship” very much anymore in academia. Gregory Chaitin: And people don’t write books. In the past, some wonderful mathematicians like G. H. Hardy (1877–1947, pictured in 1927) would write wonderful books like A Mathematician’s Apology (1940)…

red-and-blue-spiral-fractal-background-image-illustration-vortex-repeating-spiral-pattern-symmetrical-repeating-geometric-patterns-abstract-background-stockpack-adobe-stock.jpg
Red and Blue Spiral Fractal Background Image, Illustration - Vortex repeating spiral pattern, Symmetrical repeating geometric patterns. Abstract background

Mathematics: Did We Invent It Or Did We Merely Discover It?

What does it say about our universe if the deeper mathematics has always been there for us to find, if we can?

In this week’s podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin (pictured) on how math presents us with a challenging philosophical question: Does math image deep truth about our universe? Or do we just make up these math rules in our own minds to help us understand nature? https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 00:39 min. A partial transcript, Show Notes, and Additional Resources follow. Gregory Chaitin: Deep philosophical questions have many answers, sometimes contradictory answers even, that different people believe in. Some mathematics, I think, is definitely invented, not discovered. That tends to be trivial mathematics — papers that fill in much-needed gaps because…

call-for-papers-cfp-science-stockpack-adobe-stock.jpg
Call for papers (Cfp, science)

From Nature: A New, Topflight Computer Science Journal

Starting in January 2021, it proposes to tackle a key problem in computer use in science - replication of findings

The Springer Nature Group is launching a new online-only journal,Nature Computational Science. It is described as a “dedicated home for computational science” and we are told: Recent advances in computer technology, be it in hardware or in software, have revolutionized the way researchers do science: problems that are too complex for human or analytical solutions are now easier to address; problems that would take years to solve can now be unraveled in days, hours, or even seconds. The use and development of advanced computing capabilities to analyse and solve scientific problems, also known as computational science, has undoubtedly played a key role in transformational scientific breakthroughs of our last century, making progress possible in many different disciplines. Elizabeth Hawkins, “A…

worried-doctors-and-medical-researchers-on-conference-meeting-discussing-possible-solutions-for-resolving-a-world-health-crisis-health-and-medical-care-concept-selective-focus-stockpack-adobe-stock.jpg
Worried doctors and medical researchers on conference meeting, discussing possible solutions for resolving a world health crisis. Health and medical care concept. Selective focus.

Computers Excel at Finding Temporary Patterns

Which contributes to the replication crisis in science

The scientific method calls for the rigorous testing of plausible theories, ideally through randomized controlled trials. For example, a study of a COVID-19 vaccine might give the vaccine to 10,000 randomly selected people and a placebo to another 10,000, and compare the infection rates for the two groups. If the difference in the infection rates is too improbable to be explained by chance, then the difference is deemed statistically significant. How improbable? In the 1920s, the great British statistician, Sir Ronald Fisher , said that he favored a 5 percent threshold. So 5 percent became the Holy Grail. Unfortunately, the establishment of a 5 percent hurdle for statistical significance has had the perverse effect of encouraging researchers to do whatever…

tablet-displaying-scan-of-brain-activity-stockpack-adobe-stock.jpg
Tablet displaying scan of brain activity

Brain Scans Can Read Your Mind—in a Dozen Conflicting Ways

A recent study involving 70 research groups identified sharp limitations in the value of brain imaging (fMRI) in understanding the mind

In the 1990s, functional magnetic resonance imaging (fMRI) — imaging the brain in action via blood flow—seemed like a dream come true. Medical and social science researchers who flocked to use it are not going to be happy with a recent study of its limitations: There was little meaningful agreement among seventy research teams from around the world about what their results meant. In an article aptly titled “Seventy Teams of Scientists Analysed the Same Brain Data, and It Went Badly,” a neuroscientist fills us in: The group behind the Nature paper set a simple challenge: they asked teams of volunteers to each take the same set of fMRI scans from 108 people doing a decision-making task, and use them…

journal-papers.jpg
Stack of papers isolated on white background

Einstein’s Single Journal Paper Ended WWII

Does that mean that a thousand papers could multiply the effect? Think again.

It was Albert Einstein’s work on matter and energy, captured in e = mc2 that enabled the atomic bomb that ended World War II. Modern anonymous peer review today works well except that it is muddied with bias, incompetence, and ignorance. The review processes of Einstein’s day were better. A renowned expert’s approval was sufficient for a paper’s publication.1 The current system has only been in force since the end of World War II when pressure was applied to professors to write papers. The mantra “publish or perish” looks to have been coined soon after the war in 1951 by Marshall “The Medium is the Message” McLuhan.2 Earlier, professors were often discouraged from publishing. Karl Popper, one of the most…

The concept of planet Earth similar to the COVID-19 virus
The concept of planet Earth similar to the COVID-19 virus

Twenty Years on, Aliens Still Cause Global Warming

Over the years, the Jurassic Park creator observed, science has drifted from its foundation as an objective search for truth toward political power games

In 2003, author and filmmaker Michael Crichton (1942–2008), best known for Jurassic Park, made a now-famous speech at Caltech, titled “Aliens Cause Global Warming.” The title was humorous but the content was serious. He was not addressing some strange theory of global warming; he was warning about the politicization of science. Crichton (left, in 2002, courtesy Jon Chase, Harvard CC 3.0), noted that, over the years, science has drifted away from its foundation as an objective search for truth and given itself over to political power games. The first time that he witnessed that was with the famous Drake Equation, used to turn SETI speculations about space aliens into a science. The Drake equation was a series of probabilities multiplied…

rejected-concept-stockpack-adobe-stock.jpg
REJECTED CONCEPT

Einstein’s Only Rejected Paper

It was the only one reviewed anonymously, as is the practice today

Today’s collection of scholarly literature is exploding in quantity and deteriorating in quality. One solution is to return to review practices at the time of Einstein. The reviewers were much better qualified and were not anonymous.

Read More ›
Citation

Anti-Plagiarism Software Goof: Paper Rejected for Repeat Citations

The scholar was obliged by discipline rules to cite the flagged information repetitively

Not only was Jean-François Bonnefon’s paper rejected by conventional anti-plagiarism software but the rejection didn’t make any sense. Bonnefon, research director at Toulouse School of Economics, was informed of “a high level of textual overlap with previous literature” (plagiarism) when he was citing scientists’ affiliations, standard descriptions, and papers cited by other—information he was obliged to cite accurately, according to a standard format. “It would have taken two [minutes] for a human to realise the bot was acting up,” he wrote on Twitter. “But there is obviously no human in the loop here. We’re letting bots make autonomous decisions to reject scientific papers.” Reaction to the post by Dr Bonnefon, who is currently a visiting scientist at the Massachusetts Institute…