The case against peer review.

Health and medicine explained.
April 5 2005 2:03 PM

Quality Control

The case against peer review.

(Continued from Page 1)

More sweeping changes have also been suggested. Well-trained, full-time editors and professional statisticians might be able to perform the functions of peer review on their own. Or scientists en masse might be recruited: Paul Ginsparg, who runs a digital archive for unpublished physics papers, has suggested that putting "preprints" of scientific papers on the Web could let the community as a whole decide which papers are most useful. Unpublished work could be tracked by an objective measure—like how often it's cited or downloaded—and then passed along for formal publication. Government funders like the NIH could hire professional reviewers to evaluate grants, or they could replace grants with cash prizes for successful research.

When journal editors are asked about these ideas, they often quote Winston Churchill's line, "Democracy is the worst form of government except all those other forms that have been tried from time to time." Or rather, they quote other journal editors quoting that line. But it's a poor analogy, since few alternatives to peer review have been tried in modern times. And democracy isn't really a good description of peer review, either. Sure, peer review allows scientists to participate in a system of self-governance. But wouldn't BMJ's policy of open review or Ginsparg's proposal for Web-published preprints be far more democratic?

Advertisement

So far, though, the Churchill quoters are winning. A good study of peer review will take more than a small band of outsiders working part time with little money. We need a coordinated effort by scientific journals, funding agencies, and research scientists. It wouldn't be that expensive. But the funds have not been forthcoming. Instead, the federal government recently proposed using peer review "to improve the quality, objectivity, utility, and integrity of information" that it gives to the publicfor things like FDA drug requirements and HHS dietary guidelines. Ironically, most scientists hated the idea  when it was first presented; the original version of the proposal stated that only researchers from private industry, who don't receive federal funding, would get to participate. (That provision has since been eliminated. *) But forget arguing over the details of peer review. Let's first figure out whether it works.

Correc tion, April 8, 2005: The original version of this column missated a provision of the federal government's proposal on peer review. The revised version of the proposal would allow federally funded scientists to participate.

TODAY IN SLATE

The World

The Budget Disaster that Sabotaged the WHO’s Response to Ebola

Are the Attacks in Canada a Sign of ISIS on the Rise in the West?

PowerPoint Is the Worst, and Now It’s the Latest Way to Hack Into Your Computer

Is It Offensive When Kids Use Bad Words for Good Causes?

Fascinating Maps Based on Reddit, Craigslist, and OkCupid Data

Culturebox

The Real Secret of Serial

What reporter Sarah Koenig actually believes.

Culturebox

The Actual World

“Mount Thoreau” and the naming of things in the wilderness.

In Praise of 13th Grade: Why a Fifth Year of High School Is a Great Idea

A Comically Inane Court Opinion Just Upheld Puerto Rico’s Gay Marriage Ban

  News & Politics
Politics
Oct. 23 2014 3:55 PM Panda Sluggers Democrats are in trouble. Time to bash China.
  Business
Moneybox
Oct. 23 2014 5:53 PM Amazon Investors Suddenly Bearish on Losing Money
  Life
Outward
Oct. 23 2014 5:08 PM Why Is an Obscure 1968 Documentary in the Opening Credits of Transparent?
  Double X
The XX Factor
Oct. 23 2014 11:33 AM Watch Little Princesses Curse for the Feminist Cause
  Slate Plus
Working
Oct. 23 2014 11:28 AM Slate’s Working Podcast: Episode 2 Transcript Read what David Plotz asked Dr. Meri Kolbrener about her workday.
  Arts
Brow Beat
Oct. 23 2014 5:08 PM What Happens When You Serve McDonald’s to Food Snobs and Tell Them It’s Organic
  Technology
Future Tense
Oct. 23 2014 5:53 PM South Florida’s Desperate Secession Movement
  Health & Science
Science
Oct. 23 2014 5:42 PM Seriously, Evolution: WTF? Why I love the most awkward, absurd, hacked-together species.
  Sports
Sports Nut
Oct. 20 2014 5:09 PM Keepaway, on Three. Ready—Break! On his record-breaking touchdown pass, Peyton Manning couldn’t even leave the celebration to chance.