If we want to improve peer review, we'll need to invest in training
New reviewers are anxious to get some formal coaching before they start commenting the work of fellow academics. A blog article published by Elaine Devine via The Guardian. Accessed on Oct 10, 2017.
You’ve just accepted your first invitation to peer review a journal article. The paper is the culmination of years of the author’s research and its publication is an important next step. You want to get it right; give constructive feedback, check whether it works for the journal, and generally improve the quality of the article. But how can you achieve those objectives without the right training and support?
We’ve been analysing data emerging from our year-long research project on peer review. First we focused on what academics would like to see change in the process. Now we’ve examined how new reviewers feel about embarking on reviewing, and it seems clear that many would like significantly more guidance on how to get it right.
Alongside issues of transparency, much of the criticism of peer review revolves around the quality of individual reviews – from the “short and snitty” (as one researcher put it in a focus group we held) to those that may be biased by the reviewer’s personal views.
The majority of reviewers learn on the job, plunged into assessing their peers’ work as a natural progression of their academic career. This will need fixing if everyone’s overall experience of the peer review process is to improve. Here are some of the key findings from our research on the topic:
New academics want to review but don’t know how to start
Research we conducted with more than 6,300 authors of journal articles, peer reviewers and journal editors revealed that over two-thirds of researchers who have never peer reviewed a paper would like to. Of that group (drawn from the full range of subject areas) more than 60% said they would like the option to attend a workshop or formal training on peer reviewing. At the same time, over two-thirds of journal editors told us that it is difficult to find reviewers.
Guidance is not always available
For those who are already reviewing papers, the most common source of support they received was journal editor guidelines. Just under half of respondents – 45% in science, technology and medicine (STM) and 49% in the humanities and social sciences (HSS) – said they had received these. Guidelines issued by the publisher came second, with 41% of reviewers receiving them in STM, and 35% in HSS.
Only one in 10 of reviewers had been to a workshop or formal training but 55% of sciences researchers and 51% of humanities and social science researchers said they would like this opportunity. And that is despite the fact that two-thirds of reviewers rated their confidence in reviewing at eight or above out of 10.
What can editors do?
Editors are trying to recruit and retain good reviewers. But views were split as to the best way to achieve this. Some 46% of editors in STM wanted to offer formal training or workshops but 44% said they didn’t want to. In the humanities and social sciences, just 38% wanted to offer this option, with 56% saying they wouldn’t.
In terms of what help they were offering, they listed (in the following order): offering their own guidelines and advice, signposting to publisher guidelines, encouraging supervisor involvement, and, for a small number, workshops or formal training.
From this research, there seems to be a discrepancy between the support those doing the reviewing wish for and what those overseeing the process are prepared to offer. However, the lack of enthusiasm for offering workshops or formal training has made me wonder whether this may have been more popular had we made it explicit in our question that this training could be delivered by someone other than journal editors.
Why researchers want to review
Playing a part as a member of the academic community, reciprocating the benefit, and enjoying being able to help improve papers were the top three motivations for doing a peer review.
We also looked at this data across the age groups, expecting to see some differences in this order, particularly for those at the start of their career who might have been more motivated to take the job for career reasons. But, no matter what the age group, playing a part as a member of the academic community scored highest.
Understandably, those aged between 20 and 29 gave the highest score compared to other groups in the “enhancing your reputation or furthering your career” category. However, this still came fifth out of the eight possible responses for those in their twenties. And in terms of whether academics should receive payment for doing reviews, those in the youngest age group (20-29) were most in favour, while those who were over 60 were the most resistant. Whether this is due to the financial pressures for those starting out, or a possible shift in attitude towards payment for reviewers, is yet to be seen.
With peer review remaining so central to the research process, despite its flaws, understanding the motivations to publish in peer reviewed journals and to review another academic’s work is crucial. Addressing what training is needed is even more important. There’s a clear desire from the academic community to see effective support put in place, so now is the time for action.
This article was originally posted on The Guardian by Elaine Devine. View puplication.