This series of posts are authored by some of the team members at Altmetric, a data science company that provides attention data to authors, publishers, institutions and funders.
The San Francisco Declaration on Research Assessment (DORA) points out that using the Journal Impact Factor as a proxy measure for the value or quality of specific research and individual scientists leads to biased research assessment. How can we resist misusing metrics?
The Independent Review of the Role of Metrics in Research Assessment and Management was set up in April 2014 to investigate the current and potential future roles that quantitative indicators can play in the assessment and management of research. Its report, ‘The Metric Tide’, was published in July 2015 and is available below.
House of Commons Science and Technology Committee
Peer review in scholarly publishing, in one form or another, has always been regarded as crucial to the reputation and reliability of scientific research. In recent years there have been an increasing number of reports and articles assessing the current state of peer review. In view of the importance of evidence-based scientific information to government, this report covers a detailed examination of the current peer-review system as used in scientific publications. Both to see whether it is operating effectively and to shine light on new and innovative approaches. In addition. it explores some of the broader issues around research impact, publication ethics and research integrity.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
PRE (Peer Review Evaluation)
Several market research studies have evaluated peer review in recent years. A goal of the present research, commissioned by PRE, is to extend those findings to provide insight into the indicators of quality of peer review. In January, 2016 Wicherts proposed that transparency of the peer review process may be seen as an indicator of the quality of peer review. By testing a questionnaire tool with several audiences with different methods, he concludes that the tool has promising reliability and validity in assessing transparency of the peer-review process as an indicator of peer-review quality. In this market research, we ask respondents to rate the helpfulness of several criteria based in part on Wicherts’ 14-item tool which rates the transparency of a journal’s peer review process, regardless of peer review model, open or blinded, pre-publication or post-publication.
http://www.pre-val.org/wp-content/uploads/2016/09/PRE-surveywhitepaper.pdf
House of Commons Science and Technology Committee
Peer review in scholarly publishing, in one form or another, has always been regarded as crucial to the reputation and reliability of scientific research. In recent years there have been an increasing number of reports and articles assessing the current state of peer review. In view of the importance of evidence-based scientific information to government, this report covers a detailed examination of the current peer-review system as used in scientific publications. Both to see whether it is operating effectively and to shine light on new and innovative approaches. In addition. it explores some of the broader issues around research impact, publication ethics and research integrity.
http://www.publications.parliament.uk/pa/cm201012/cmselect/cmsctech/856/856.pdf
PRE (Peer Review Evaluation)
Several market research studies have evaluated peer review in recent years. A goal of the present research, commissioned by PRE, is to extend those findings to provide insight into the indicators of quality of peer review. In January, 2016 Wicherts proposed that transparency of the peer review process may be seen as an indicator of the quality of peer review. By testing a questionnaire tool with several audiences with different methods, he concludes that the tool has promising reliability and validity in assessing transparency of the peer-review process as an indicator of peer-review quality. In this market research, we ask respondents to rate the helpfulness of several criteria based in part on Wicherts’ 14-item tool which rates the transparency of a journal’s peer review process, regardless of peer review model, open or blinded, pre-publication or post-publication.
http://www.pre-val.org/wp-content/uploads/2016/09/PRE-surveywhitepaper.pdf
The Royal Society
The Science as an open enterprise report highlights the need to grapple with the huge deluge of data created by modern technologies in order to preserve the principle of openness and to exploit data in ways that have the potential to create a second open science revolution.
Exploring massive amounts of data using modern digital technologies has enormous potential for science and its application in public policy and business. The report maps out the changes that are required by scientists, their institutions and those that fund and support science if this potential is to be realised.
https://royalsociety.org/topics-policy/projects/science-public-enterprise/report/
Helping students wandering in the land of the mind to work, finish, publish, become famous, have impact, win prizes, and reflect glory on their thesis advisors
