The San Francisco Declaration on Research Assessment (DORA) points out that using the Journal Impact Factor as a proxy measure for the value or quality of specific research and individual scientists leads to biased research assessment. How can we resist misusing metrics?
The studies presented in this section investigated dissemination to the public from the researchers’ and public engagement enablers’ point of view. The first, “Factors affecting public engagement by researchers” came out of a project by the UK consortium TNS-BMRB & PSI, and represents an update to an earlier project by the Royal Society in 2006.
Richard D. Morey, Christopher D. Chambers, Peter J. Etchells, Christine R. Harris, Rink Hoekstra, Daniel Lakens, Stephan Lewandowsky, Candice Coker Morey, Daniel P. Newman, Felix D. Schonbrodt, Wolf Vanpaemel, Eric-Jan Wagenmakers, Rolf A. Zwaan
Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.
Adrian Mulligan, Louise Hall, Ellen Raphael
This large-scale international study measures the attitudes of more than 4,000 researchers toward peer review. In 2009, 40,000 authors of research papers from across the globe were invited to complete an online survey. Researchers were asked to rate a number of general statements about peer review, and then a subset of respondents, who had themselves peer reviewed, rated a series of statements concerning their experience of peer review. The study found that the peer review process is highly regarded by the vast majority of researchers and considered by most to be essential to the communication of scholarly research. Nine out of 10 authors believe that peer review improved the last paper they published. Double-blind peer review is considered the most effective form of peer review. Nearly three quarters of researchers think that technological advances are making peer review more effective. Most researchers believe that although peer review should identify fraud, it is very difficult for it to do so. Reviewers are committed to conducting peer review in the future and believe that simple practical steps, such as training new reviewers would further improve peer review.
Richard D. Morey, Christopher D. Chambers, Peter J. Etchells, Christine R. Harris, Rink Hoekstra, Daniël Lakens, Stephan Lewandowsky, Candice Coker Morey, Daniel P. Newman, Felix D. Schönbrodt, Wolf Vanpaemel, Eric-Jan Wagenmakers, Rolf A. Zwaan
Openness is one of the central values of science. Open scientific practices such as sharing data, materials and analysis scripts alongside published articles have many benefits, including easier replication and extension studies, increased availability of data for theory-building and meta-analysis, and increased possibility of review and collaboration even after a paper has been published. Although modern information technology makes sharing easier than ever before, uptake of open practices had been slow. We suggest this might be in part due to a social dilemma arising from misaligned incentives and propose a specific, concrete mechanism—reviewers withholding comprehensive review—to achieve the goal of creating the expectation of open practices as a matter of scientific principle.
Carole J. Lee, Cassidy R. Sugimoto, Guo Zhang, Blaise Cronin
Peer review is the cornerstone of scholarly publishing and it is essential that peer reviewers are appointed on the basis of their expertise alone. However, it is difficult to check for any bias in the peer-review process because the identity of peer reviewers generally remains confidential. Here, using public information about the identities of 9000 editors and 43000 reviewers from the Frontiers series of journals, we show that women are underrepresented in the peer-review process, that editors of both genders operate with substantial same-gender preference (homophily), and that the mechanisms of this homophily are gender-dependent. We also show that homophily will persist even if numerical parity between genders is reached, highlighting the need for increased efforts to combat subtler forms of gender bias in scholarly publishing.
Mario Biagioli
Together with tenure, peer review is probably the most distinctive feature of the modern academic system. Peer review, we are told, sets academia apart from all other professions by construing value through peer judgment, not market dynamics. Given the remarkable epistemological and symbolic burden placed on peer review, it is surprising to find that so little research has analyzed it either empirically (in its actual daily practices) or philosophically (as one of the conditions of possibility of academic knowledge). While academics discuss it quite frequently, they do not frame it as an intellectual subject. Instead, they either confine it to private conversations or treat it as one of the practical aspects of the profession. Typically, peer review comes up in the context of personal complaints about the perceived incompetence (or other unflattering traits) of editors and referees. But when the dust settles, it is not uncommon to hear appreciative remarks for the referees’ time-consuming and unpaid contributions, or to see them thanked in the acknowledgments.
Changes in scholarly publishing have resulted in a move toward openness. To this end, new, open models of peer review are emerging. While the scholarly literature has examined and discussed open peer review, no established definition of it exists, nor are there uniform implementations of open peer review processes.
The traditional forms of scientific publishing and peer review do not live up to all demands of efficient communication and quality assurance in today’s highly diverse and rapidly evolving world of science.
Helping students wandering in the land of the mind to work, finish, publish, become famous, have impact, win prizes, and reflect glory on their thesis advisors
