Science

Peer Review Is a Thankless Task

So what motivates the scientists who do it the most? We asked them.

peer review.
The goal of peer review is to ensure that only the most scientifically sound research papers make it into scientific journals.

Jesussanz/Thinkstock

When Jonas Ranstam wakes up in the morning, one of the first things he does is review other people’s scientific papers. By his count, the independent medical statistician spends 24 hours a week peer reviewing—and in sum, over the past year, he has reviewed 661 papers. By one count, this is the most of any other scientific reviewer, and the dedication has won him the grand prize in the inaugural Sentinel of Science Awards. The prize, which involved a small amount of cash and is sponsored by a handful publishers, is being presented Friday by Publons, a platform that tracks reviewers’ progress. (Ranstam uses it himself.)

In addition to Ranstam, the Sentinel of Science Awards is also honoring top reviewers by country and field. The company hopes the awards will provide some recognition for what is often a thankless task. (It’s also a promotional tactic—there’s an award for “evangelist editors” who have referred the most people to the platform.) But recognizing peer review is a worthy cause: The labor it takes to peer review science papers would cost a couple billion dollars a year, if the reviewers were paid for their time, which they almost never are.

The goal of peer review is to ensure that only the most scientifically sound research papers make it into scientific journals—a tall order for a system that operates almost entirely on scientists’ goodwill and dedication. By Publons’ count, the average reviewer referees just 4.4 papers per year—though even that number might be high, as it only includes the people enthusiastic enough to register for the account that helps them track this work.

So what motivates the super peer reviewers to do it? I collected thoughts from 22 winners of the Publons award through a (unscientific) Google survey to find out (for several of them, their dedication takes a full day each week). Nearly all of the respondents indicated that they felt it was their duty to help further science, and 80 percent agreed that they find peer reviewing helps them stay on top of their field. Just four attributed their reviewing to their institution’s encouragement. Only two agreed that, in part, they just like critiquing things anonymously. A couple friends, who are likely less established than my award-winning survey respondents, told me that being asked to review is a recognition that you’re an expert in a field, which is kind of an honor.

Ranstam says he is happy to receive an award, but that recognition is not what compels him to do the work. “Being a reviewer and an author are two sides of the same coin,” he told me. “You’re morally obliged. That’s how the system works.”

Other winners echoed his sentiments. Academic disciplines are communities, said Stephen Turner, a philosophy professor at the University of South Florida who Publons is recognizing for being a top reviewer of his field thanks to the more than 30 papers he processed this year.* “Reviewing is just something one does as a caring member of that community,” he wrote in the Google survey.

Grigorios Kyriakopoulos, a chemical engineer who received the second overall prize with 437 reviews, noted that offering input on a study makes him feel like a co-creator. “I deeply appreciate their [the authors’] laborious work, and my motives of reviewing are always to provide objective and constructive review comments that will maximize the novelty of all studies’ reviewed,” he told me in an email.*

What’s clear is that we need more thoughtful reviews from qualified people. Ranstam is also a deputy editor of a journal, Osteoarthritis and Cartilage, and in this role, he says he has a hard time finding enough people like him and his fellow awardees to take on papers.

Increasing the sheer number of papers scientists are willing to review would help fix the problem, but in order to properly review a paper you should have some subject-area expertise. So the pool of super-willing participants would also have to reflect the same swath of topics in the millions of papers that are published every year, which is a tricky task. Statisticians like Ranstam are already in highest demand, which is why he sometimes receives a small honorarium for reviewing a paper—not a common occurrence.

The problem is figuring out the right way to incentivize people to spend their mornings reviewing other people’s work. Money doesn’t seem like a viable option, since people don’t really want to pay for scientific papers in the first place. Perhaps rewards like this could bring some prestige to an otherwise thankless task, but there are only so many prizes you can give out before the power is diluted.

It turns out the top reviewers themselves might have some suggestions: Hosam Elbaz, the top reviewer in pharmacology, toxicology, and pharmaceutics, suggested via my Google form survey giving peer review more weight in promotions or tenure consideration. Pradeep Muragundi, the winner in the category of health professions, suggested that peer reviewers get credit when a paper they referred is cited, much the same way that researchers do, to incentivize quality work.

Both could be good options that would give these selfless peer reviewers a break.

*Correction, Sept. 26, 2016: This story originally misstated how many papers Stephen Turner reviewed; it was more than 30, not exactly 30. (Return.)

*Update, Sept. 26, 2016: This sentence has been updated to note whose work was appreciated by Kyriakopoulos. (Return.)