Peer Review Nightmares—and How to Fix a Broken System

Getting schooled.
July 15 2014 11:32 PM

Revise and Resubmit!

Peer review is slow. It’s unhelpful. It’s generally awful. Here’s how to fix it.

140715_EDU_PeerReview

Illustration by Rob Donnelly

I’m sure you’ve heard the adage “publish or perish.” In order to get tenure—or, as the abysmal job market begets hyper-professionalization, to be considered for a job at all—a scholar must have a certain amount of articles appear in “peer-reviewed” academic journals. These journals—Obscure Subfield Quarterly, One-Word Pretentious Greek Thing, etc.—usually have a circulation of about 300. Their subscriptions cost hundreds—if not thousands—of dollars per year. Thus, they remain all but inaccessible to someone without research-university affiliation. Thus those articles upon which careers depend are usually read by exactly three people.

Rebecca Schuman Rebecca Schuman

Rebecca Schuman is an education columnist for Slate.

I’m not even going to get in to the particulars of why this intentionally restricted readership helps no one (except the occasional for-profit publishing conglomerate)—because that’s not even the worst thing about academic publishing. The worst thing about academic publishing is two-thirds of a given article’s three-person audience: The “referees,” or “peer reviewers,” to whom the editor sends the piece for vetting.

Think of your meanest high school mean girl at her most gleefully, underminingly vicious. Now give her a doctorate in your discipline, and a modicum of power over your future. That’s peer review.

Advertisement

Should all academic articles just be accepted with a hug and a cookie? Of course not. A thorough vetting of a new piece of scholarship is indeed crucial—but right now, rather than being constructively critical, far too many peer reviews are just cruel for no reason (or, apparently, fraudulent!). But some articles are just bad! whine the meanies, in a panic that they’ll lose their only consequence-free opportunity to express their professional misery. And yet, you can reject an article without stating, definitively, that its author has no business in the profession. I’ve done it. It’s not even the slightest bit difficult.

Sure, in an ideal situation, peer reviewers are collegial, constructive experts in the author’s sub-field with a legitimate interest in making an article better. But most situations are far from ideal: The “peer” is actually a put-upon grad student or recent Ph.D., fresh off of seven years of being kicked in the gut and just aching to do some kicking of his own. Peer review is seen as a thankless chore with little reward other than the editor’s someday owing you a favor—thus, articles regularly, I’m talking almost always, languish untouched on the referee’s desk for six, 12, even 18 months. Come to think of it, I’m actually still waiting to hear about a submission from 2012. (Spoiler alert: I don’t care anymore.)

Some journal articles are accepted on their first pass. (I wrote one that was! Will you be the fourth person to read it?) Many more are rejected with extensive commentary, and more still receive a special place in academic purgatory: a directive to “revise and resubmit” according to the referee’s requirements. All of this, again, is perfectly fine in theory. Some such suggestions are even helpful—but all too often, they’re hidden amidst the venting of petty vendettas and pettier agendas. I’ve actually been quite fortunate with academic publishing—I’ve even got a book coming out next year. But a quick perusal of my peers’ experiences lets me know I’ve been unusually lucky:

So many readers’ reports can be boiled down to: “Why wasn’t this article exactly the one I would have written?” (Or: “Why wasn’t I cited enough?”) Thus, too often, academic peer review is gatekeeping for its own sake, whose chief result is to wound the author as deeply and existentially as possible. For scholarly articles are exacting, they’re difficult, and they are often an expression of one’s full academic self. They can take months—sometimes years—to write. They’re our guts, spilled onto the paper, the absolute best work we can possibly do. Peer reviewers know this—they write articles themselves—and yet they still seize it as a rare opportunity to kick down.

Aside, however, from imploring academics not to be jerks (a likely futile task), the usual fixes suggested (and, sometimes, implemented), aren’t bad—but they could be better. Some academics argue, for example, that if reviewers signed their names, they might think twice about their pettiest jabs. I’m not so sure, though. Anyone who’s sat through the Q&A after a conference presentation knows that a lot of academics consider it sporting to rip each other’s faces off in public (“I don’t have a question so much as a comment”; a 20-minute tirade ensues).

Another solution is the idea of open-source peer review, of the sort scholar Kathleen Fitzpatrick recently accomplished with her book Planned Obsolescence: Publishing, Technology and the Future of the Academy. The 43 peer reviewers she crowdsourced online, Fitzpatrick tells me, came from a diverse variety of disciplines, and thus “often disagreed with one another, and their discussions gave me important information about ways my work was being interpreted.” This, she said, combined with the two traditional reviews from her publisher (which she also says were quite constructive), improved her book drastically. I’m glad this system worked for her, but I admire her fortitude—because when I read “crowdsourced peer review,” I hear “Internet comments,” and I want to crawl into a hole and die.

What we need is an idea that takes advantage of, rather than battles against, academics’ petty self-interest. Luckily, the peers who review your work may think of themselves as above you in every way—but there’s one important manner in which they’re not: They need to publish too.

Right now anyone can submit an article or book to any journal or press, and if the beleaguered (often unpaid) editor likes it, she begs friends or grad students or total strangers to look at it for peer review. But what if in order to be eligible to submit an academic article to a journal, a scholar had first to volunteer to review someone else’s article for that same journal? What if that review had only two requirements: It has to be timely (in academese, by the way, this means three months). And that review has to be constructive. You want to publish and not perish? First you have to earn that right by making a punctual, non-petty investment into the publishing enterprise. Journals get better, more motivated reviewers; authors are more invested in actually reading and contributing the journals. Everybody wins. Call it “peer review review.”

  Slate Plus
Tv Club
Oct. 30 2014 7:27 PM The American Horror Story: Freak Show Podcast, Episode 4 The “Edward Mordrake (Part 2)” edition.