Meet OpenUP’s Pilot Studies

In May this year our pilot studies started. After a half year we concluded the first implementation phase and published our interim results report. In this post we want to present you the seven pilot studies more in detail. The first pilot study (Pilot No 1) tests an open peer review workflow in a conference setting. For this purpose, we adapted and are currently evaluating an existing conference management system (CMS) that includes features such as:

• open report: the review report is published alongside the publication,
• open identity: authors and/or reviewers are aware of each other’s identity, or
• open pre-review: early versions of material are public before the review.

The adapted CMS tool has been already applied in a conference. A second test will be done at another conference next year. The adapted open source solution is available on GitHub. Through the increased transparency the review process will set new quality standards.  

Pilot study number two (Pilot No 2) investigates the applicability of (open) peer review to research data in disciplines related to Social Sciences. In a first phase, we analysed current dataset management and diffusion practices in this field. In a second phase, we will conduct interviews and a survey taking into account both perspectives by data providers/journal publishers and data users. The aim is to identify best practices of data repositories and attitudes of data users. OpenUP will test pre-publication and post-publication criteria and capture trends and needs to be addressed in the Social sciences to improve data evaluation.

The goal of the pilot study number three (Pilot No 3) is to define a framework for a data journal in the Humanities and to provide a related action plan. The pilot investigates how quality assessment and (open) peer review can be applied to research data in the Humanities. In a first phase current practices and requirements of the Humanities data community have been analysed, and the Humanities data field mapped based on existing research. In a second phase these insights will be complemented by additional interviews with stakeholders from the research community. OpenUP will identify best practices in sharing and publishing data in arts and humanities, and develop practical guidance for researchers for every step of the data publishing process.

The fourth OpenUP pilot study (Pilot No 4) investigates whether qualitative research, in particular data analysis and data collection, can be transferred to open online groups. In a first step we adapted an existing open online research software prototype, which enables collaborative online data analysis and collection. The technology enables citizens from most diverse backgrounds to participate in data collection and analysis in qualitative research. In a second step, it is foreseen to engage citizens through a Massive Open Online Research (MOOR) course. Through this collaborative research environment OpenUP seeks to engage citizens in open research.

Pilot study number five (Pilot No 5) analyses and tests how disseminated research results can be made more interesting, appealing, and usable for business and general public target audiences. In a first step we created recommendations and guidelines for reaching out to these audiences. In a second step the guidelines will be applied and tested by a smart city project.

The goal of the sixth pilot study (Pilot No 6) is to analyse how much the use of new dissemination practices and channels (such as blogs or social media) depends on established cultures of appreciation that reward usage of these channels. To achieve this we engaged with two research communities to analyse their use of social media tools and Altmetrics. Stakeholders of both communities have been involved in interviews and meetings. In the next phase they will be asked to report their social media and Altmetrics activities on a regular basis. The goal is to create a community profile on Altmetrics and evaluate the applicability of Altmetrics in this specific context.  

Finally, Pilot Study No 7 conducts a quantitative, statistical and econometric analysis of large scale datasets on the supply of and demand for scholarly works on various illegal platforms (e.g. Sci-Hub, Library Genesis). The pilot will model which works-specific factors may explain the availability and illegal demand for individual works (e.g. price, legal availability). In a first phase datasets from piratical access providers have been enriched and an online service enabling real-time exploration of illegally downloaded books has been developed. In the next phase the data will be analysed using advanced statistical modelling methods and geospatial models to explain the impact of legal (un)availability of scientific works. The results will shed light on any potential failures or shortcomings on the legal access channels (markets and libraries) that drive people to use the black markets.