Mind Matters Natural and Artificial Intelligence News and Analysis

TagPeer Review

stack-of-papers-isolated-on-white-background-stockpack-adobe-stock
Stack of papers isolated on white background

A Vulnerable System: Fake Papers and Imaginary Scientists

Publication counts and citation indexes are too noisy and too easily manipulated to be reliable

In the last two posts, we examined how scientific publication has ceased to be a good measure of scientific accomplishment, and how the peer review system is being gamed by unscrupulous publishers and researchers alike. Now, we will continue the discussion on the undermining of scientific publication using two examples: SCIgen and citation counts. SCIgen In 2005 three MIT graduate computer science students created a prank program they called SCIgen for using randomly selected words to generate bogus computer-science papers complete with realistic graphs of random numbers. Their goal was “maximum amusement rather than coherence,” and also to demonstrate that some academic conferences will accept almost anything. They submitted a hoax paper titled, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,”…

lying-businessman-holding-fingers-crossed-behind-his-back-stockpack-adobe-stock
Lying businessman holding fingers crossed behind his back

Gaming the System: The Flaws in Peer Review

Peer review is well-intentioned, but flawed in many ways

Last time, we examined how scientific publication has ceased to be a good measure of scientific accomplishment because it has now become a target, following Goodhart’s Law. In today’s post, we will continue that examination by turning to the peer review system, and how that system is being gamed by unscrupulous publishers and researchers alike. In theory, the peer review process is intended to ensure that research papers do not get published unless impartial experts in the field deem them worthy of publication. Peer review is well-intentioned, but flawed in many ways. First, the best researchers are incredibly busy and naturally more inclined to do their own research than to review someone else’s work. Thus, peer review is often cursory or…

custom-library-stockpack-adobe-stock
Custom library

Publish or Perish — Another Example of Goodhart’s Law

In becoming a target, publication has ceased to be a good measure

The linchpin of scientific advances is that scientists publish their findings so that others can learn from them and expand on their insights. This is why some books are rightly considered among the most influential mathematical and scientific books of all time:  Elements, Euclid, c. 300 B.C.Physics, Aristotle, c. 330 B.C.On the Revolutions of Heavenly Spheres, Nicolaus Copernicus, 1543Dialogue Concerning the Two Chief World Systems, Galileo Galilei, 1632Mathematical Principles of Natural Philosophy, Isaac Newton, 1687The Origin of Species, Charles Darwin, 1859 As Newton said, “If I have seen further it is by standing on the shoulders of Giants.” It seems logical to gauge the importance of modern-day researchers by how much they have published and how often their research has been cited…

black-and-caucasian-female-scientists-have-work-related-discussion-male-researcher-joins-them-they-work-in-a-modern-laboratory-research-center-stockpack-adobe-stock.jpg
Black and Caucasian Female Scientists have Work Related Discussion, Male Researcher Joins Them. They Work in a Modern Laboratory/ Research Center.

Is the Research Money for AI Spent at Universities Just Wasted?

A computer engineering prof tells an AI think tank about the time a four-star general visited his university

In a recent podcast, Walter Bradley Center director Robert J. Marks spoke with Robert D. Atkinson and Jackie Whisman at the prominent AI think tank, Information Technology and Innovation Foundation, about his recent book, The Case for Killer Robots—a plea for American military brass to see that AI is an inevitable part of modern defense strategies, to be managed rather than avoided. The book may be downloaded free here. In this third part (here are Part 1 and Part 2), the discussion turned to stalled AI research at universities: https://episodes.castos.com/mindmatters/Mind-Matters-109-Robert-Marks.mp3 The discussion of the state of AI research begins at 17:48 (A portion of the transcript follows. The whole transcript is here. Notes and links follow below.) Rob Atkinson: You…

rejected-concept-stockpack-adobe-stock.jpg
REJECTED CONCEPT

Einstein’s Only Rejected Paper

It was the only one reviewed anonymously, as is the practice today

Today’s collection of scholarly literature is exploding in quantity and deteriorating in quality. One solution is to return to review practices at the time of Einstein. The reviewers were much better qualified and were not anonymous.

Read More ›