Past Projects

Journal Observatory

Time: 2022-2023
Funding: Dutch Research Council (NWO)
Role: Partner

Keywords: Scholarly Communication Policies, Open Science

ORRG Project Contact: Tony Ross-Hellauer

As new models of publishing such as Publish-Review-Curate, publication as you go, preprint review and others emerge, distinct publishing functions like dissemination and evaluation are increasingly decoupled. This creates the need for different platforms to interact and at least to be aware of each other’s policies and requirements. At present, there are minimal standards to enable the systematic interoperability of these platforms. At the research output level, standards like Docmaps and COAR Notify are under development. However, to empower further innovation in scholarly communication, a shared way to describe these different platforms and their possibilities of interaction is required. To address these above challenges, the Journal Observatory project aims:

1. To define an extensible, machine-readable and traceable way to describe the policies and practices of the various platforms involved in disseminating and evaluating scholarly works: the Scholarly Communication Platform Framework.
2. To demonstrate the value of this new framework by building a demonstration prototype called the Journal Observatory, a resource which combines data on journals and other publication platforms from various sources to clarify policy information for authors, reviewers and others.

Evaluation study on the implementation of cross-cutting issues in Horizon 2020

Time: 2022
Client: Directorate General for Research and Innovation (DGRTD, EC)
Role: Partner

Keywords: EC, Horizon 2020, Cross-Cutting Issues, Widening Participation, RRI

ORRG Project Contact: Tony Ross-Hellauer

The final report for the project can be found here.

This study evaluated cross-cutting issues in Horizon 2020 (such as interdisciplinarity, sustainability, and international cooperation, among others). Developing suggestions on how to define, implement and monitor such overarching priorities in future programmes was also one aim of the study. Twelve cross-cutting issues were investigated each in a small case study with a diverse set of methods. The ORRG took the lead on the case studies on widening participation across Europe and on responsible research and innovation (RRI).

Assessing the Reproducibility of Research Results in EU Framework Programmes for Research

Time: 2020-2022
Client: Directorate General for Research and Innovation (DGRTD, EC)
Role: Partner

Keywords: EC, Horizon 2020, Horizon Europe, Reproducibility, Research Quality

ORRG Project Contact: Tony Ross-Hellauer

The final report for the project can be found here.

The core issue the study addressed is that research results published today are often impossible to reproduce. The lack of reproducibility has serious negative effects on the performance of the research and innovation system. It also affects citizens’ trust in science.

The study assisted the Contracting Authority to understand, test and monitor the progress of reproducibility over time and across the programmes, as a direct and/or indirect response to a range of policy interventions to increase the wider availability of results (reproducibility as strictly defined above and open science more in general). Furthermore, the study assisted directly with the gradual introduction of the principle and practices of reproducibility in EU-funded research and innovation. The study was completed through the implementation of five tasks:

Task 1: Assess the overall quality and reproducibility of projects and programmes
Task 2. Assess the impact of measures to increase reproducibility (including predicting reproducibility from applications)
Task 3. Assess the actual reuse of existing data in funded projects under H2020 and HE
Task 4. Assess the effects of interventions on reproducibility on trust in science
Task 5. Determine the overall implication of the study for policy action on reproducibility and provide actionable recommendations

The overall design of the study followed a ‘blinded’ approach where two separate teams collected data and compared the results. Under the first approach, our team of experts collected quantitative data for all 1000 projects and their outputs and individuals. Under the second track, researchers scrutinised the 50 projects in great detail qualitatively (assessing the projects by undertaking the expert review).

ORRG was leading evidence synthesis and development of recommendations to the EC, as well as contributing heavily to the qualitative assessment of project outputs and researcher/editor/funder attitudes.

Observing and Negating Matthew Effects in Responsible Research

Time: 2019-2022
Funding: Horizon 2020 (EC)
Role: Lead

Keywords: Open Science, Cumulative Advantage, Inequality

ORRG Project Contact: Tony Ross-Hellauer

Recommendations for Research Institutions, Funders and Researchers

All outputs can be found on Zenodo.

ON-MERRIT aimed to investigate inequalities in the uptake of Open Science. Open Science promises to make scientific research more inclusive, understandable to the public, and accessible to and reusable for large audiences. However, making science open to the general public stands the risk of being undermined by a dynamic of cumulative advantage – those who already have stand to gain even more through Open Science. ON-MERRIT – Observing and Negating Matthew Effects in Responsible Research and Innovation Transition recognised this threat to be urgent. Using a mix of sociological, bibliometric and computational approaches, the project investigated how existing inequalities along dimensions such as gender, geographical location or institutional standing drive outcomes in the uptake of Open Science and Responsible Research and Innovation across academia, industry and policy-making. ON-MERRIT gathered a range of skills in a consortium funded by the EU Horizon 2020 programme including experts in open science, data analytics, interaction with data, policy research, as well as stakeholder engagement.

ORRG took the function as coordinating partner of this interdisciplinary expert consortium, which included Know-Center (AT), Open University (UK), University of Goettingen (DE), University of Minho (PT) and Graz University of Technology (AT). Moreover, ORRG contributed to reviews and surveys of the uptake of open science resources in industry and policy-making, and the analyses of relationships between Open Science practices and academic performance, research training as well as institutional promotion criteria. Based on these results, ORRG finally used agent-based modeling to test the effect of various policies and incentives, and compile a set of policy-recommendations.