This is a blog item, published by WIRED, which is about the story of a neuroscientist named Niko Kriegeskorte, a cognitive neuroscientist at the Medical Research Council in the UK who, since December 2015, has performed all of his peer review openly. That means he publishes his reviews as he finishes them on his personal blog—sharing on Twitter and Facebook, too—before a paper is even accepted.
Anthony Ross-Hellauer
This is part two of a series of posts describing OpenAIRE’s work to find a community-endorsed definition of “open peer review” (OPR), its features and implementations. As described in Part One, OpenAIRE collected 122 definitions of “open review” or “open peer review” from the scientific literature. Iterative analysis of these definitions resulted in the identification of seven distinct OPR traits at work in various combinations amongst these definitions:
Anthony Ross-Hellauer
At present, there is neither a standardized definition of “open peer review” (OPR) nor an agreed schema of its features and implementations, which is highly problematic for discussion of its potential benefits and drawbacks. This new series of blog posts reports on work to resolve these difficulties by analysing the literature for available definitions of “open peer review” and “open review”. In all, 122 definitions have been collected and codified against a range of independent OPR traits, in order to build a coherent typology of the many different adaptations to the traditional peer review that has come to be signified by the term OPR and hence provide a unified definition.
This initiative shares a vision of an independent, democratic academic evaluation model free from the conflicts of interest imposed by the agendas of journals and their commercial publishers. It aims to promote complementary strategies to comprise the ingredients needed to attain this goal and to encourage scholars and interested parties to experiment with new modes that can assist the transition to free, independent, open and transparent peer review. In addition, it considers that any platform developed to implement free and open peer review should be independent of intermediaries. To mitigate potential conflicts of interest such platforms should ideally be under the management of an open community, be open source and operate in a non-profit manner.
House of Commons Science and Technology Committee
Peer review in scholarly publishing, in one form or another, has always been regarded as crucial to the reputation and reliability of scientific research. In recent years there have been an increasing number of reports and articles assessing the current state of peer review. In view of the importance of evidence-based scientific information to government, this report covers a detailed examination of the current peer-review system as used in scientific publications. Both to see whether it is operating effectively and to shine light on new and innovative approaches. In addition. it explores some of the broader issues around research impact, publication ethics and research integrity.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
Adrian Mulligan, Louise Hall, Ellen Raphael
This large-scale international study measures the attitudes of more than 4,000 researchers toward peer review. In 2009, 40,000 authors of research papers from across the globe were invited to complete an online survey. Researchers were asked to rate a number of general statements about peer review, and then a subset of respondents, who had themselves peer reviewed, rated a series of statements concerning their experience of peer review. The study found that the peer review process is highly regarded by the vast majority of researchers and considered by most to be essential to the communication of scholarly research. Nine out of 10 authors believe that peer review improved the last paper they published. Double-blind peer review is considered the most effective form of peer review. Nearly three quarters of researchers think that technological advances are making peer review more effective. Most researchers believe that although peer review should identify fraud, it is very difficult for it to do so. Reviewers are committed to conducting peer review in the future and believe that simple practical steps, such as training new reviewers would further improve peer review.
PRE (Peer Review Evaluation)
Several market research studies have evaluated peer review in recent years. A goal of the present research, commissioned by PRE, is to extend those findings to provide insight into the indicators of quality of peer review. In January, 2016 Wicherts proposed that transparency of the peer review process may be seen as an indicator of the quality of peer review. By testing a questionnaire tool with several audiences with different methods, he concludes that the tool has promising reliability and validity in assessing transparency of the peer-review process as an indicator of peer-review quality. In this market research, we ask respondents to rate the helpfulness of several criteria based in part on Wicherts’ 14-item tool which rates the transparency of a journal’s peer review process, regardless of peer review model, open or blinded, pre-publication or post-publication.
http://www.pre-val.org/wp-content/uploads/2016/09/PRE-surveywhitepaper.pdf
The Research Information Network
This guide has been produced by The Research Information Network to provide researchers with an understanding of the peer review process and some of the current issues surrounding the debate about peer review.
Diane Harley, Sophia Krzys Acord, Sarah Earl-Novell, Shannon Lawrence, C. Judson King
Copyright: Center for Studies in Higher Education, UC Berkley
Since 2005, the Center for Studies in Higher Education (CSHE), has been conducting research to understand the needs and practices of faculty for in-progress scholarly communication (i.e., forms of communication employed as research is being executed) as well as archival publication. This report brings together the responses of 160 interviewees across 45, mostly elite, research institutions in seven selected academic fields: archaeology, astrophysics, biology, economics, history, music, and political science. The overview document summarizes the main practices explored across all seven disciplines: tenure and promotion, dissemination, sharing, collaboration, resource creation and consumption, and public engagement. In this report, readers can search various topics within and across case studies. The report identifies five key topics, addressed in detail in the case studies, that require real attention:
Council of Science Editors
Peer review is the principal mechanism by which the quality of research is judged. Most funding decisions in science and the academic advancement of scientists are based on peer-reviewed publications. Because the number of scientific articles published each year continues to grow, the quality of the peer-review process and the quality of the editorial board are cited as primary influences on a journal’s reputation, impact factor, and standing in the field. Scientific journals publishing peer-reviewed articles depend heavily on the scientific referees or reviewers who typically volunteer their time and expertise. In most circumstances, at least 2 reviewers are solicited to evaluate a manuscript; some journals request 3 reviews. This may be required in situations where review by a statistician is needed. In cases of controversy or strong disagreement regarding the merits of the work, an additional review may also be solicited or one of the journal’s editors might give an evaluation. More than 3 reviewers are sometimes used if reviewers from several fields are needed to obtain a thorough evaluation of a paper. In addition to fairness in judgment and expertise in the field, peer reviewers have significant responsibilities toward authors, editors, and readers.
