Assessing the reproducibility of research results in EU Framework Programmes for Research

The core issue the study will address is that research results published today are often impossible to reproduce. The lack of reproducibility has serious negative effects on the performance of the research and innovation system. It also affects citizens’ trust in science.

The study will assist the Contracting Authority to understand, test and monitor the progress of reproducibility over time and across the programmes, as a direct and/or indirect response to a range of policy interventions to increase the wider availability of results (reproducibility as strictly defined above and open science more in general). Furthermore, the study will assist directly with the gradual introduction of the principle and practices of reproducibility in EU-funded research and innovation. The study will be completed through the implementation of five tasks:

• Task 1: Assessment of the overall quality and reproducibility of projects and programmes
• Task 2. Assess the impact of measures to increase reproducibility (including predicting reproducibility from applications)
• Task 3. Assess the actual reuse of existing data in funded projects under H2020 and HE
• Task 4. Assess the effects of interventions on reproducibility on trust in science
• Task 5. Determine the overall implication of the study for policy action on reproducibility and provide actionable recommendations

The overall design of the study will follow a ‘blinded’ approach where two separate teams collect data and compare the results. Under the first approach, our team of experts will collect quantitative data for all 1000 projects and their outputs and individuals. Under the second track, researchers will scrutinise the 50 projects in great detail qualitatively (assessing the projects by undertaking the expert review).

ORRG is leading evidence synthesis and development of recommendations to the EC, as well as contributing heavily to the qualitative assessment of project outputs and researcher/editor/funder attitudes.

The final report for the project can be found here.