OpenUP Pilot 2: “Open Peer Review for Research Data”

The objective of pilot 2 “Open Peer Review for Research Data” is to investigate the applicability of (open) peer review to datasets in disciplines related to Social Sciences. The pilot study aims at identifying strong and weak elements in the dataset review and validation process and outline best practices that facilitate transparency of the process as well as data dissemination, reliability and reuse. In most disciplines, data peer review is at its initial stage. It is mainly being formally performed in data journals, sometimes also applying open peer review. Another important form of data validation is carried out by data repositories. When data are submitted to disciplinary and trusted repositories, the quality control performed can be considered a form of peer review before publication. This form of pre-publication review is then verified through formal citation and/or statistics of use by the data users in the post-publication phase. Our analysis of the current dataset management and dissemination practices in Social Sciences showed that there is a limited propensity to data sharing and lack of data journals that do foster data open peer review. Social scientists often rely on official statistical data for their research. Moreover, the presence of trusted repositories represents a benchmark, to which other initiatives of data dissemination can be compared.

The pilot 2 research team, after considering the involvement of different research communities in Social Sciences, decided to formalize the collaboration with the Human Mortality Database (HMD). HMD is a well-known data source established in 2000 that provides open access to data in an important and specific demography topic. It was developed by a small group of researchers who aimed at diffusing an innovative method to measure population aging. HMD gradually developed from earlier research projects. The collaborative organization set up to disseminate these data is also worth analyzing in a pilot as it can provide insights on best practices that can be adopted also by other communities. Moreover, HMD has numerous data users worldwide, who belong to different scientific communities as well as to the business sector. Therefore, the part of the pilot focusing on data users has good chances to be promising. The reached agreement with HMD specified the following steps to be taken by the two institutions in context of this pilot study:

• In-depth interviews with HMD management and Data country agents who are responsible for the validation of data coming from the contributing countries.
• Development of a common survey to HMD users.
• Support the implementation of additional tools for the HMD website, such as: Google analytics to have statistics on access, view, downloads; implementation of tools (Hypothesis, Disqus, etc.) to have feedback from users.

In addition, a gender review on the survey questionnaire wording was conducted as well as a gender balanced interview plan defined. For more on HMD project check out: http://www.mortality.org/