Peer Review Week 2017: what do our referees think?

By Tom Sharp on September 15, 2017

Peer Review Week 2017 is all about transparency. Here at IOP, we recognize the invaluable service provided by our referees. We want your feedback, and as such, we’ve been asking our referees about the peer review process, and how we can provide the best possible service for our community.

We’ve collected responses from 166 referees so far. Here are a few things that we’ve learned so far. Firstly; who are our reviewers and why do they do it?

Figure 1: Responses to “On average, how many times do you complete journal reviews each year?”

Figure 2: Number of responses to the question “What motivates you to accept an invitation to review an article?”.

It’s clear that you referees are a committed bunch, with 42% of respondents reviewing 5-14 articles per year, and 15% taking on 15 or more. What motivates you to give up that much time? 82% do it out of interest in the article, and 68% because of the reputation of the journal. Only 17% of referees do it for credit or recognition, while 52% believe it is an expectation of the academic community. It’s clear that our referees understand the vital role of peer review within scientific literature.

We also want to know how you think we should do peer review. IOP journals use a single blind approach, whereby referees remain anonymous but the author’s identity is visible to referees. Recently, we have added the option of double blind peer review to our Materials Research Express and Biomedical Physics & Engineering Express. We asked our referees which method they preferred, including post-publication, collaborative, and open peer review.

Figure 3: Response to the question “Which of the following forms of peer review is your preferred method? Single blind: the reviewer is aware of the author’s identity, but the author is not made aware of who has reviewed their article. Double blind: the reviewers of the paper do not know the identity of the authors, and the authors do not know the identity of the reviewers. Post publication: online comments in the published version of the article are allowed, independent of any formal peer review that may have already occurred on the article. Collaborative: reviewers interact with each other and can access each other’s reports, either during review or immediately afterwards. Open: all reviewer names and review reports are published with the article.”

A whopping 65% of you selected single blind, with 24% preferring double blind. 56% of respondents said they would be willing to participate double blind peer review. It’s great to know our referees approve of the way we do things! Interestingly, since we started offering double blind peer review as an option, about 20% of authors have elected to use it; very much in line with the results of our poll.

It’s essential for us as a publisher to know how we can improve our support for referees. High quality peer review is obviously vital, and we want to help our referees provide the best possible service to authors and readers. You can find our current guidelines for referees here. We asked what aspects of peer review our referees would like more training on.

Figure 4: Number of responses to “What, if any, aspects of peer review would you like more training on?”

It’s clear that many people (57%) would like clarification of journal quality standards. Providing constructive feedback, and working with editors, also received a large number of votes.

We’d like to thank all our referees for the fantastic service and feedback, which we’ll be using to improve the way we do peer review in the future.

For more information about peer review week, click here.

Watch the journey of a paper through peer review here.

This work is licensed under a Creative Commons Attribution 3.0 Unported License