OpenUP policy recommendations No 1 (Peer Review)
OpenUP synthesised and validated key project results and derived 5 recommendations to foster the take-up of novel practices in scholarly peer review, research dissemination & assessment. Discover the rational behind the Recommendation #1
Rationale
The traditional peer review process used by many scientific journals is subject to criticism on several fronts.[1] It has been described as unreliable, too lengthy, biased and lacking accountability and transparency.[2] On the other hand, OPR and its different components,[3] open up the traditional process in order to help address these criticisms. OPR introduces such aspects as open identities (author and reviewer disclose their identities), open reports (review reports are published alongside the publication) and open participation (wider participation of interested parties in the process).[4] These new aspects enable a more open discussion on research outputs. OPR can also help authors get feedback not only on the final research outputs (articles or books) but also on what is created at various stages of the research process (software codes or datasets). Various initiatives are emerging among researchers who are willing to employ OPR. For example, the Open Science Peer Review Oath[5] includes four principles[6] that are in line with OPR practices. Reviewers are encouraged to add a link to the oath and to inform authors and potential publishers that the reviewer will follow the principles set out in the oath. Also, alternatives to the traditional closed peer review process are already offered by a number of open access journals and publication platforms (Frontiers, BMJ and F1000research).
OpenUP’s survey results revealed that despite the varying levels of agreement among researchers to different peer review aspects, “review of data underlying an article” received the strongest support.[7] Over 70% of respondents “strongly supported” or were “rather in support” of this aspect of OPR. On the other hand, Pilot 2 found that OPR of data in social sciences is rarely adopted.[8] Also, Pilot 3 found that data review includes more issues to manage as datasets can be complex, multifaceted information objects that constantly change over time[9]. The other traits of OPR, such as open identity, open participation, open report, and open pre-review were supported by a lower number of survey respondents.[10] Interview data showed a division of opinions on whether the use of OPR should be taken up more widely.[11] On the other hand, feedback on the OPR process applied to conferences given by the researchers involved in Pilot 1 was positive. Overall, the participants expressed a strong acceptance of the proposed OPR process and would support it again. The participants’ greatest fears associated with OPR included: biased/whitewashed reviews due to non-anonymity; backlash for bad reviewing (e.g. over other channels/private email); and added effort and risk for reviews outside one’s own expertise (lay-man reviews). Some respondents of the OpenUP survey saw the need to fund and promote studies as well as to advertise successful examples in the context of OPR to showcase the good examples. Others emphasized lack of awareness among researchers and more trainings and incentives needed to increase the adoption of OPR practices.
Recommendation #1 - Specific Actions - Relevant Stakeholders
Recommendation |
Specific actions |
Responsible stakeholders |
Run pilots that implement OPR practices to generate evidence |
|
EU and national policy makers Research funders Publishers Institutional administration |
MORE Information
OpenUP synthesised and validated key project results and derived five recommendations to foster the take-up of novel practices in scholarly peer review, research dissemination and assessment while considering existing gaps in evidence and disciplinary differences. To find out more about the OpenUP policy recommendations, follow the link.
References
[1] See Görögh, E., et al. (2017). Deliverable D3.1– Practices, evaluation and mapping: Methods, tools and user needs. OpenUP project.
[2] Lee, C. J., Sugimoto, C. R., Zhang, G. and Cronin, B. (2013). Bias in peer review. J Am Soc Inf Sci Tec, Vol. 64, PP: 2–17. https://doi.org/10.1002/asi.22784; Manchikanti, L., Kaye, A. D., Boswell, M and Hirsch, J. A. (2015). Medical Journal Peer Review: Process and Bias. Pain Physician, vol. 18, pp: E1-E14.
[3] Ross-Hellauer, T. (2017). What is open peer review? A systematic review. F1000Research, 6:588. https://doi.org/10.12688/f1000research.11369.2
[4] European Commission. (2015). Validation of the Results of the Public Consultation on Science 2.0: Science in Transition. http://ec.europa.eu/research/consultations/science-2.0/science_2_0_final_report.pdf#view=fit&pagemode=none.
[5] Aleksic, J. et al. (2015). An Open Science Peer Review Oath. F1000Research, 3:271. https://doi.org/10.12688/f1000research.5686.2.
[6] Principle 1: I will sign my name to my review; Principle 2: I will review with integrity; Principle 3: I will treat the review as a discourse with you; in particular, I will provide constructive criticism; Principle 4: I will be an ambassador for the practice of open science.
[7] Görögh, E., et al. (2017). Deliverable D3.1– Practices, evaluation and mapping: Methods, tools and user needs. OpenUP project.
[8] Forthcoming: OpenUP. (2018). D6.3 Final Use case evaluation report.
[9] Carpenter, T. (2017). What Constitutes Peer Review of Data? A Survey of Peer Review Guidelines. Scholarly Kitchen. Retrieved 5 August 2018 from: https://scholarlykitchen.sspnet.org/2017/04/11/what-constitutes-peer-review-research-data/
[10] Banelyte, V. et al. (2017). Deliverable D7.2 – Completed Policy review and mapping and field research activities. OpenUP project.
[11] ibid.