Peer Review Nightmares—and How to Fix a Broken System

Getting schooled.
July 15 2014 11:32 PM

Revise and Resubmit!

Peer review is slow. It’s unhelpful. It’s generally awful. Here’s how to fix it.

140715_EDU_PeerReview

Illustration by Rob Donnelly

I’m sure you’ve heard the adage “publish or perish.” In order to get tenure—or, as the abysmal job market begets hyper-professionalization, to be considered for a job at all—a scholar must have a certain amount of articles appear in “peer-reviewed” academic journals. These journals—Obscure Subfield Quarterly, One-Word Pretentious Greek Thing, etc.—usually have a circulation of about 300. Their subscriptions cost hundreds—if not thousands—of dollars per year. Thus, they remain all but inaccessible to someone without research-university affiliation. Thus those articles upon which careers depend are usually read by exactly three people.

Rebecca Schuman Rebecca Schuman

Rebecca Schuman is an education columnist for Slate.

I’m not even going to get in to the particulars of why this intentionally restricted readership helps no one (except the occasional for-profit publishing conglomerate)—because that’s not even the worst thing about academic publishing. The worst thing about academic publishing is two-thirds of a given article’s three-person audience: The “referees,” or “peer reviewers,” to whom the editor sends the piece for vetting.

Think of your meanest high school mean girl at her most gleefully, underminingly vicious. Now give her a doctorate in your discipline, and a modicum of power over your future. That’s peer review.

Advertisement

Should all academic articles just be accepted with a hug and a cookie? Of course not. A thorough vetting of a new piece of scholarship is indeed crucial—but right now, rather than being constructively critical, far too many peer reviews are just cruel for no reason (or, apparently, fraudulent!). But some articles are just bad! whine the meanies, in a panic that they’ll lose their only consequence-free opportunity to express their professional misery. And yet, you can reject an article without stating, definitively, that its author has no business in the profession. I’ve done it. It’s not even the slightest bit difficult.

Sure, in an ideal situation, peer reviewers are collegial, constructive experts in the author’s sub-field with a legitimate interest in making an article better. But most situations are far from ideal: The “peer” is actually a put-upon grad student or recent Ph.D., fresh off of seven years of being kicked in the gut and just aching to do some kicking of his own. Peer review is seen as a thankless chore with little reward other than the editor’s someday owing you a favor—thus, articles regularly, I’m talking almost always, languish untouched on the referee’s desk for six, 12, even 18 months. Come to think of it, I’m actually still waiting to hear about a submission from 2012. (Spoiler alert: I don’t care anymore.)

Some journal articles are accepted on their first pass. (I wrote one that was! Will you be the fourth person to read it?) Many more are rejected with extensive commentary, and more still receive a special place in academic purgatory: a directive to “revise and resubmit” according to the referee’s requirements. All of this, again, is perfectly fine in theory. Some such suggestions are even helpful—but all too often, they’re hidden amidst the venting of petty vendettas and pettier agendas. I’ve actually been quite fortunate with academic publishing—I’ve even got a book coming out next year. But a quick perusal of my peers’ experiences lets me know I’ve been unusually lucky:

So many readers’ reports can be boiled down to: “Why wasn’t this article exactly the one I would have written?” (Or: “Why wasn’t I cited enough?”) Thus, too often, academic peer review is gatekeeping for its own sake, whose chief result is to wound the author as deeply and existentially as possible. For scholarly articles are exacting, they’re difficult, and they are often an expression of one’s full academic self. They can take months—sometimes years—to write. They’re our guts, spilled onto the paper, the absolute best work we can possibly do. Peer reviewers know this—they write articles themselves—and yet they still seize it as a rare opportunity to kick down.

Aside, however, from imploring academics not to be jerks (a likely futile task), the usual fixes suggested (and, sometimes, implemented), aren’t bad—but they could be better. Some academics argue, for example, that if reviewers signed their names, they might think twice about their pettiest jabs. I’m not so sure, though. Anyone who’s sat through the Q&A after a conference presentation knows that a lot of academics consider it sporting to rip each other’s faces off in public (“I don’t have a question so much as a comment”; a 20-minute tirade ensues).

Another solution is the idea of open-source peer review, of the sort scholar Kathleen Fitzpatrick recently accomplished with her book Planned Obsolescence: Publishing, Technology and the Future of the Academy. The 43 peer reviewers she crowdsourced online, Fitzpatrick tells me, came from a diverse variety of disciplines, and thus “often disagreed with one another, and their discussions gave me important information about ways my work was being interpreted.” This, she said, combined with the two traditional reviews from her publisher (which she also says were quite constructive), improved her book drastically. I’m glad this system worked for her, but I admire her fortitude—because when I read “crowdsourced peer review,” I hear “Internet comments,” and I want to crawl into a hole and die.

What we need is an idea that takes advantage of, rather than battles against, academics’ petty self-interest. Luckily, the peers who review your work may think of themselves as above you in every way—but there’s one important manner in which they’re not: They need to publish too.

Right now anyone can submit an article or book to any journal or press, and if the beleaguered (often unpaid) editor likes it, she begs friends or grad students or total strangers to look at it for peer review. But what if in order to be eligible to submit an academic article to a journal, a scholar had first to volunteer to review someone else’s article for that same journal? What if that review had only two requirements: It has to be timely (in academese, by the way, this means three months). And that review has to be constructive. You want to publish and not perish? First you have to earn that right by making a punctual, non-petty investment into the publishing enterprise. Journals get better, more motivated reviewers; authors are more invested in actually reading and contributing the journals. Everybody wins. Call it “peer review review.”

Rebecca Schuman is an education columnist for Slate.

TODAY IN SLATE

History

The Self-Made Man

The story of America’s most pliable, pernicious, irrepressible myth.

Michigan’s Tradition of Football “Toughness” Needs to Go—Starting With Coach Hoke

Does Your Child Have “Sluggish Cognitive Tempo”? Or Is That Just a Disorder Made Up to Scare You?

The First Case of Ebola in America Has Been Diagnosed in Dallas

Windows 8 Was So Bad That Microsoft Will Skip Straight to Windows 10

Politics

Mad About Modi


Why the controversial Indian prime minister drew 19,000 cheering fans to Madison Square Garden.


Building a Better Workplace

You Deserve a Pre-cation

The smartest job perk you’ve never heard of.

Don’t Panic! The U.S. Already Stops Ebola and Similar Diseases From Spreading. Here’s How.

Parents, Get Your Teenage Daughters the IUD

The XX Factor
Sept. 30 2014 12:34 PM Parents, Get Your Teenage Daughters the IUD
  News & Politics
Politics
Sept. 30 2014 6:59 PM The Democrats’ War at Home Can the president’s party defend itself from the president’s foreign policy blunders?
  Business
Moneybox
Sept. 30 2014 7:02 PM At Long Last, eBay Sets PayPal Free
  Life
Gaming
Sept. 30 2014 7:35 PM Who Owns Scrabble’s Word List? Hasbro says the list of playable words belongs to the company. Players beg to differ.
  Double X
The XX Factor
Sept. 30 2014 12:34 PM Parents, Get Your Teenage Daughters the IUD
  Slate Plus
Behind the Scenes
Sept. 30 2014 3:21 PM Meet Jordan Weissmann Five questions with Slate’s senior business and economics correspondent.
  Arts
Brow Beat
Sept. 30 2014 8:54 PM Bette Davis Talks Gender Roles in a Delightful, Animated Interview From 1963
  Technology
Future Tense
Sept. 30 2014 7:00 PM There’s Going to Be a Live-Action Tetris Movie for Some Reason
  Health & Science
Medical Examiner
Sept. 30 2014 6:44 PM Ebola Was Already Here How the United States contains deadly hemorrhagic fevers.
  Sports
Sports Nut
Sept. 30 2014 5:54 PM Goodbye, Tough Guy It’s time for Michigan to fire its toughness-obsessed coach, Brady Hoke.