Category: Peer Review | Page 3

Stack of papers isolated on white background

^{Type
post

Gary Smith
Date
June 17, 2021

Categorized
Peer Review, Science

Tagged
citation counts, expert systems, Google Scholar, Peer Review, SCIgen}

A Vulnerable System: Fake Papers and Imaginary Scientists

_{Publication counts and citation indexes are too noisy and too easily manipulated to be reliable} _{Gary Smith
June 17, 2021

7

Peer Review, Science}

In the last two posts, we examined how scientific publication has ceased to be a good measure of scientific accomplishment, and how the peer review system is being gamed by unscrupulous publishers and researchers alike. Now, we will continue the discussion on the undermining of scientific publication using two examples: SCIgen and citation counts. SCIgen In 2005 three MIT graduate computer science students created a prank program they called SCIgen for using randomly selected words to generate bogus computer-science papers complete with realistic graphs of random numbers. Their goal was “maximum amusement rather than coherence,” and also to demonstrate that some academic conferences will accept almost anything. They submitted a hoax paper titled, “Rooter: A Methodology for the Typical Unification of Access Points and Redundancy,” Read More ›

Lying businessman holding fingers crossed behind his back

^{Type
post

Gary Smith
Date
June 15, 2021

Categorized
Peer Review, Science

Tagged
Goodhart's Law, Peer Review}

Gaming the System: The Flaws in Peer Review

_{Peer review is well-intentioned, but flawed in many ways} _{Gary Smith
June 15, 2021

9

Peer Review, Science}

Last time, we examined how scientific publication has ceased to be a good measure of scientific accomplishment because it has now become a target, following Goodhart’s Law. In today’s post, we will continue that examination by turning to the peer review system, and how that system is being gamed by unscrupulous publishers and researchers alike. In theory, the peer review process is intended to ensure that research papers do not get published unless impartial experts in the field deem them worthy of publication. Peer review is well-intentioned, but flawed in many ways. First, the best researchers are incredibly busy and naturally more inclined to do their own research than to review someone else’s work. Thus, peer review is often cursory or Read More ›

^{Type
post

Gary Smith
Date
June 14, 2021

Categorized
Peer Review, Science

Tagged
Google Scholar, Peer Review, publish or perish}

Publish or Perish — Another Example of Goodhart’s Law

_{In becoming a target, publication has ceased to be a good measure} _{Gary Smith
June 14, 2021

5

Peer Review, Science}

The linchpin of scientific advances is that scientists publish their findings so that others can learn from them and expand on their insights. This is why some books are rightly considered among the most influential mathematical and scientific books of all time: Elements, Euclid, c. 300 B.C.Physics, Aristotle, c. 330 B.C.On the Revolutions of Heavenly Spheres, Nicolaus Copernicus, 1543Dialogue Concerning the Two Chief World Systems, Galileo Galilei, 1632Mathematical Principles of Natural Philosophy, Isaac Newton, 1687The Origin of Species, Charles Darwin, 1859 As Newton said, “If I have seen further it is by standing on the shoulders of Giants.” It seems logical to gauge the importance of modern-day researchers by how much they have published and how often their research has been cited Read More ›

^{Type
post

Robert J. Marks
Date
May 13, 2021

Categorized
Media, Peer Review, Science

Tagged
Featured, Google Scholar, Institute of Electrical and Electronic Engineers (IEEE)), Scholarly Publishing}

Inside the Economics of Science Papers

_{Here’s an inside look at who pays if you read for free} _{Robert J. Marks
May 13, 2021

6

Media, Peer Review, Science}

When a scholarly paper is published, someone has to pay. Publishers like Institute of Electrical and Electronic Engineers (IEEE), my professional society, and Springer charge big bucks to read their papers. The fees are billed to individual subscribers and, more commonly, to companies and universities who want to give their employees access to the papers. My own university, Baylor, like most research universities, has a considerable library budget on account of these publisher fees. There is growing pressure to kill these fees in favor of “open access” to scholarly papers. Thus, anyone can read a scholarly paper at any time for free. Free access takes the money from the pockets of publishers so they push back. Someone has to pay, Read More ›

^{Type
post

News
Date
March 23, 2021

Categorized
Mathematics, Peer Review, Science

Tagged
Elon Musk, Featured, Gregory Chaitin, IBM (and innovation), Robert J. Marks}

Gregory Chaitin on How Bureaucracy Chokes Science Today

_{He complains, They’re managing to make it impossible for anybody to do any real research. You have to say in advance what you’re going to accomplish. You have to have milestones, reports} _{News
March 23, 2021

10

Mathematics, Peer Review, Science}

In last week’s podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin on how Stephen Wolfram’s software has taken much of the drudgery out of math. At the same time, in Chaitin’s view, a threat looms: A new, more bureaucratic, mindset threatens to take the creativity out of science, technology, and math: https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 19:45 min. A partial transcript, Show Notes, and Additional Resources follow. Robert J. Marks: I was sitting down tallying, I think, the intellectual giants that have introduced new mathematical ideas, brand new. I was thinking of people like Claude Shannon, Lotfi Zadeh, yourself… I don’t know if we Read More ›

^{Type
post

News
Date
March 20, 2021

Categorized
Mathematics, Peer Review

Tagged
Andrew Wiles, Elon Musk, Featured, G. H. Hardy (1877-1947), Gregory Chaitin, Karl Popper, publish or perish, Robert J. Marks, Srinivasa Ramanujan, William Sealy Gosset}

Why Don’t We See Many Great Books on Math Any More?

_{Decades ago, Gregory Chaitin reminds us, mathematicians were not forced by the rules of the academic establishment to keep producing papers, so they could write key books.} _{News
March 20, 2021

9

Mathematics, Peer Review}

In our most recent podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin on many things mathematical, including whether math is invented or discovered. This time out, Chaitin talks about why he thinks great books on math, advancing new theorems, aren’t written much any more: https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 02:49 min. A partial transcript, Show Notes, and Additional Resources follow. Robert J. Marks: You don’t hear the word “scholarship” very much anymore in academia. Gregory Chaitin: And people don’t write books. In the past, some wonderful mathematicians like G. H. Hardy (1877–1947, pictured in 1927) would write wonderful books like A Mathematician’s Apology (1940) Read More ›

Red and Blue Spiral Fractal Background Image, Illustration - Vortex repeating spiral pattern, Symmetrical repeating geometric patterns. Abstract background

^{Type
post

News
Date
March 19, 2021

Categorized
Mathematics, Peer Review, Philosophy

Tagged
Featured, Gregory Chaitin, publish or perish, Robert J. Marks}

Mathematics: Did We Invent It Or Did We Merely Discover It?

_{What does it say about our universe if the deeper mathematics has always been there for us to find, if we can?} _{News
March 19, 2021

7

Mathematics, Peer Review, Philosophy}

In this week’s podcast, “The Chaitin Interview III: The Changing Landscape for Mathematics,” Walter Bradley Center director Robert J. Marks interviewed mathematician and computer scientist Gregory Chaitin (pictured) on how math presents us with a challenging philosophical question: Does math image deep truth about our universe? Or do we just make up these math rules in our own minds to help us understand nature? https://episodes.castos.com/mindmatters/Mind-Matters-126-Gregory-Chaitin.mp3 This portion begins at 00:39 min. A partial transcript, Show Notes, and Additional Resources follow. Gregory Chaitin: Deep philosophical questions have many answers, sometimes contradictory answers even, that different people believe in. Some mathematics, I think, is definitely invented, not discovered. That tends to be trivial mathematics — papers that fill in much-needed gaps because Read More ›

^{Type
post

News
Date
September 21, 2020

Categorized
Artificial Intelligence, Peer Review

Tagged
Gary Smith}

From Nature: A New, Topflight Computer Science Journal

_{Starting in January 2021, it proposes to tackle a key problem in computer use in science - replication of findings} _{News
September 21, 2020

4

Artificial Intelligence, Peer Review}

The Springer Nature Group is launching a new online-only journal,Nature Computational Science. It is described as a “dedicated home for computational science” and we are told: Recent advances in computer technology, be it in hardware or in software, have revolutionized the way researchers do science: problems that are too complex for human or analytical solutions are now easier to address; problems that would take years to solve can now be unraveled in days, hours, or even seconds. The use and development of advanced computing capabilities to analyse and solve scientific problems, also known as computational science, has undoubtedly played a key role in transformational scientific breakthroughs of our last century, making progress possible in many different disciplines. Elizabeth Hawkins, “A Read More ›

Worried doctors and medical researchers on conference meeting, discussing possible solutions for resolving a world health crisis. Health and medical care concept. Selective focus.

^{Type
post

Gary Smith
Date
September 14, 2020

Categorized
Medicine and Health, Peer Review, Science

Tagged
bitcoin, Featured, James Tobin, Meaningless correlations, replication crisis, Reproducibility, Ronald Fisher, Statistical significance}

Computers Excel at Finding Temporary Patterns

_{Which contributes to the replication crisis in science} _{Gary Smith
September 14, 2020

7

Medicine and Health, Peer Review, Science}

The scientific method calls for the rigorous testing of plausible theories, ideally through randomized controlled trials. For example, a study of a COVID-19 vaccine might give the vaccine to 10,000 randomly selected people and a placebo to another 10,000, and compare the infection rates for the two groups. If the difference in the infection rates is too improbable to be explained by chance, then the difference is deemed statistically significant. How improbable? In the 1920s, the great British statistician, Sir Ronald Fisher , said that he favored a 5 percent threshold. So 5 percent became the Holy Grail. Unfortunately, the establishment of a 5 percent hurdle for statistical significance has had the perverse effect of encouraging researchers to do whatever Read More ›

Tablet displaying scan of brain activity

^{Type
post

News
Date
August 20, 2020

Categorized
Neuroscience, Peer Review

Tagged
fMRI, functional magnetic resonance imaging, Mark Humphries}

Brain Scans Can Read Your Mind—in a Dozen Conflicting Ways

_{A recent study involving 70 research groups identified sharp limitations in the value of brain imaging (fMRI) in understanding the mind} _{News
August 20, 2020

8

Neuroscience, Peer Review}

In the 1990s, functional magnetic resonance imaging (fMRI) — imaging the brain in action via blood flow—seemed like a dream come true. Medical and social science researchers who flocked to use it are not going to be happy with a recent study of its limitations: There was little meaningful agreement among seventy research teams from around the world about what their results meant. In an article aptly titled “Seventy Teams of Scientists Analysed the Same Brain Data, and It Went Badly,” a neuroscientist fills us in: The group behind the Nature paper set a simple challenge: they asked teams of volunteers to each take the same set of fMRI scans from 108 people doing a decision-making task, and use them Read More ›

^{Type
post

Robert J. Marks
Date
June 9, 2020

Categorized
Military, Peer Review

Tagged
Albert Einstein, Frank Tipler, Hiroshima, Karl Popper, Marshall McLuhan, William Sealy Gosset}

Einstein’s Single Journal Paper Ended WWII

_{Does that mean that a thousand papers could multiply the effect? Think again.} _{Robert J. Marks
June 9, 2020

5

Military, Peer Review}

It was Albert Einstein’s work on matter and energy, captured in e = mc2 that enabled the atomic bomb that ended World War II. Modern anonymous peer review today works well except that it is muddied with bias, incompetence, and ignorance. The review processes of Einstein’s day were better. A renowned expert’s approval was sufficient for a paper’s publication.1 The current system has only been in force since the end of World War II when pressure was applied to professors to write papers. The mantra “publish or perish” looks to have been coined soon after the war in 1951 by Marshall “The Medium is the Message” McLuhan.2 Earlier, professors were often discouraged from publishing. Karl Popper, one of the most Read More ›

The concept of planet Earth similar to the COVID-19 virus

^{Type
post

Jonathan Bartlett
Date
June 7, 2020

Categorized
Medicine and Health, Peer Review, Science

Tagged
censorship, COVID-19, Donald Trump, Drake Equation, Featured, Michael Crichton, The Lancet}

Twenty Years on, Aliens Still Cause Global Warming

_{Over the years, the Jurassic Park creator observed, science has drifted from its foundation as an objective search for truth toward political power games} _{Jonathan Bartlett
June 7, 2020

5

Medicine and Health, Peer Review, Science}

In 2003, author and filmmaker Michael Crichton (1942–2008), best known for Jurassic Park, made a now-famous speech at Caltech, titled “Aliens Cause Global Warming.” The title was humorous but the content was serious. He was not addressing some strange theory of global warming; he was warning about the politicization of science. Crichton (left, in 2002, courtesy Jon Chase, Harvard CC 3.0), noted that, over the years, science has drifted away from its foundation as an objective search for truth and given itself over to political power games. The first time that he witnessed that was with the famous Drake Equation, used to turn SETI speculations about space aliens into a science. The Drake equation was a series of probabilities multiplied Read More ›

^{Type
post

Robert J. Marks
Date
May 14, 2020

Categorized
Peer Review

Tagged
Albert Einstein, Featured, Frank Tipler, Peer Review, Physics, Scholarly Publishing}

Einstein’s Only Rejected Paper

_{It was the only one reviewed anonymously, as is the practice today} _{Robert J. Marks
May 14, 2020

5

Peer Review}

Today’s collection of scholarly literature is exploding in quantity and deteriorating in quality. One solution is to return to review practices at the time of Einstein. The reviewers were much better qualified and were not anonymous.

Anti-Plagiarism Software Goof: Paper Rejected for Repeat Citations

_{The scholar was obliged by discipline rules to cite the flagged information repetitively} _{News
February 6, 2020

3

Artificial Intelligence, Arts & Culture, Machine Learning, Peer Review}

Not only was Jean-François Bonnefon’s paper rejected by conventional anti-plagiarism software but the rejection didn’t make any sense. Bonnefon, research director at Toulouse School of Economics, was informed of “a high level of textual overlap with previous literature” (plagiarism) when he was citing scientists’ affiliations, standard descriptions, and papers cited by other—information he was obliged to cite accurately, according to a standard format. “It would have taken two [minutes] for a human to realise the bot was acting up,” he wrote on Twitter. “But there is obviously no human in the loop here. We’re letting bots make autonomous decisions to reject scientific papers.” Reaction to the post by Dr Bonnefon, who is currently a visiting scientist at the Massachusetts Institute Read More ›

^{Type
post

Robert J. Marks
Date
July 18, 2019

Categorized
Education, Peer Review

Tagged
academia, Campbell’s Law, citation, Featured, Goodhart’s Law, Impact factor, Longform}

Why It’s So Hard To Reform Peer Review

_{Robert J. Marks: Reformers are battling numerical laws that govern how incentives work. Know your enemy!} _{Robert J. Marks
July 18, 2019

17

Education, Peer Review}

Measurement creates a temptation to achieve a measurable goal by less than totally honest means. As in physics, the simple act of measuring invariably disturbs what you are trying to measure.