OpenUP Pilots: Synthesis and Lessons Learned

During the course of 2017-2018, OpenUP run seven pilot studies in specific settings and in close collaboration with various research communities. The seven pilots are attributed to three Use Cases, which correspond to the OpenUP pillars.

The pilots successfully connected and collaborated with communities who were particularly interested in applying open science approaches and providing alternative access to scientific output. What OpenUP did was, on the one hand, raise their interest and support them in achieving their open science goals. On the other hand, we had intensive dialogues with them, collected their problems and challenges that they encountered while putting the tested approaches into practice. Beyond testing the selected open science approaches in particular application contexts, we sustainably supported the involved communities to adopt the tested practices and solutions beyond the duration of OpenUP. In two cases, the collaboration activities with the communities resulted in ulterior funding and/or long-term cooperation between the pilot leading institutions and the community contacts (Pilot 4, Pilot 6).

During the final evaluation phase, all pilots were able to achieve the main goals previously defined:

• The two design studies (Pilot 2, Pilot 3) provided the results from the analysis of current dataset management and sharing practices in the Social Sciences, and on research data management in the Humanities. Both pilots provided a qualitative analysis of the interviews and survey on the applicability of (open) peer review to datasets in Social Sciences, and of the state-of-the-art report and requirements analysis for data journals in the Humanities.
• Pilots producing software (Pilot 1, Pilot 4) delivered testable and tested prototypes of adapted CMS software and an open online research tool. The pilots delivered a qualitative and quantitative analysis of the feedback from the participants testing the CRM tool, and of the test round of the OpenOnlineResearch tool.
• Pilots 5 and 6 delivered draft recommendations and guidelines for reaching stakeholders from the business sector and the general public; and draft community profiles of Translational Medical Research communities. Both pilots delivered a qualitative and quantitative analysis of the resulting dissemination material and impact, and of the Open Data related Altmetrics usage in translational research.
• Finally, the statistical analysis pilot (Pilot 7) delivered a report of the piratical access provider dataset enrichment, and a first visualisation of the data created with the developed online service allowing real-time exploration of the dataset. The pilot delivered a micro- and macroeconomic model of country level usage of piratical scholarly publications.

Key findings and lessons learned from the pilots include:

• Open Peer Review:

• Open Peer Review for conferences was well received by the involved communities
• Open Peer Review for data supports Open Research Data
• Principles of Open Peer Review of research data

• Open Research Data:

• Publishing, managing and reviewing research data: Recognition of and rewards for activities are needed
• Diversity of research data: Definition suitable for the Humanities needed
• Principles of research data management and data journals in the Humanities
• Fostering data publication and quality assessment: Principles for creating an Open Research Database
• Motivations for sharing research data in Social Sciences and the Humanities
• Barriers to open research data (focus: Humanities and Life Sciences/Biomedicine)

• Reaching and involving stakeholders outside of academia:

• Transferring the research workflow to the web involving the general public works
• Diverse approach for defining and supporting open science dissemination is needed

• Impact assessment:

• Using altmetrics for assessing the impact of research dissemination at specific target groups is not straightforward
• Suggested measures for establishing open research data incentives
• Use of pirated academic publications skew demand for key library resources

• Policy and action recommendations:

• Strong commitment by organisations, funders, and policy makers needed
• Support and training for researchers needed
• Encourage Open Online Research in research projects
• Lessons learned from academic piracy for shaping business models in scholarly publishing

Read the full evaluation report here.