Skip to main content

The Reform Research Assessment (RRA) and the need for Transition

What is Research Assessment

Research assessment systematically evaluates the quality, impact, rigor, and significance of scholarly work. It significantly influences crucial decisions related to funding, hiring, promotions, tenure, and institutional strategies. Traditional approaches primarily utilize quantitative metrics, such as Journal Impact Factors and citation counts, to gauge research success.

Why Reform is Necessary

Traditional research assessment practices tend to prioritize publication quantity and journal prestige, often neglecting qualitative measures and diverse scholarly outputs such as datasets, software, community engagement, mentoring, and interdisciplinary collaboration. Such practices can lead researchers to prioritize quantity over genuine quality and societal impact, thereby undermining innovation, fairness, inclusivity, and transparency in scholarly evaluation.

Multiple global initiatives have driven significant progress in reshaping research assessment, championing responsible, fair, and inclusive practices that move beyond simplistic metrics to recognize the full breadth of scholarly contribution and impact.

San Francisco Declaration on Research Assessment (DORA, 2013)
  • Advocates against using Journal Impact Factors for individual researcher evaluations.
  • Promotes evaluation of research based on the actual content and diversity of outputs (e.g., software, data, mentorship).
  • Emphasizes transparency, inclusivity, and a holistic evaluation approach.
Leiden Manifesto (2015)
  • Presents ten guiding principles for the responsible use of research metrics.
  • Reinforces the importance of expert judgment alongside appropriate quantitative metrics.
  • Encourages transparency, contextual sensitivity, and regular assessment method evaluations.
Coalition for Advancing Research Assessment (CoARA, 2022)
  • Establishes a global coalition committed to systemic reform of research assessment.
  • Balances qualitative insights with responsibly applied quantitative indicators.
  • Emphasizes accountability, transparency, inclusivity, and ethical oversight.
CoARA has established 13 operational working groups tackling the reform of research assessment from different perspectives. These include working groups on open infrastructures, multilingualism, responsible use of metrics, artificial intelligence and ethics, narrative CVs, and more. One of these groups specifically focuses on developing principles, frameworks, and tools for Open Infrastructures.

Open Infrastructures are not merely supportive tools — they are the backbone of a modern research assessment ecosystem. By offering transparent, community-governed, and interoperable systems, they create the foundational layer that allows all other reforms to take root and scale. Without them, initiatives like narrative CVs, multilingual evaluation practices, responsible metrics, and ethical AI remain fragmented and difficult to operationalize.

Their role is essential in ensuring that assessment practices are FAIR (Findable, Accessible, Interoperable, Reusable), trustworthy, and resilient over time. Open infrastructures bring coherence, scalability, and sustainability to reform efforts — making them not just a technical requirement, but a strategic necessity for meaningful and lasting change.

Open Infrastructures: A Foundation for Implementing RRA

Open Infrastructures (OIs) are essential for effectively implementing responsible research assessment, aligning with global reform initiatives like CoARA, DORA, and the Leiden Manifesto. OIs offer transparent, inclusive, interoperable, and ethically grounded environments to ensure a fairer and more holistic approach to research evaluation. The CoARA Working Group ‘’Towards Open Infrastructures for Responsible Research Assessment’’ has conducted related work in mapping the frameworks and principles or OIs fit for RRA (Open Infrastructures for Responsible Research Assessment: Principles and Framework):

Technical Robustness: Ensures data integrity, traceability, reproducibility, interoperability, and scalability. Transparent integration of advanced technologies like AI, Explainable AI (XAI), and NLP enhances accuracy and fairness.

Operational Efficiency: Offers streamlined workflows to reduce administrative burden, while valuing a wide range of scholarly contributions. Continuous capacity-building maintains stakeholder engagement and proficiency.

Community-Centred Governance: Promotes inclusive governance with active participation from diverse stakeholders, ensuring transparency, adaptability, and sustainability.

Ethical and Inclusive Practices: Maintains fairness, ethical data management, transparent technology application, and robust ethical oversight to guarantee impartial assessments.

To effectively implement responsible research assessment using open infrastructures:

  • Clear, institution-specific action plans guided by the CoARA framework.
  • Offer regular training and resources to stakeholders to ensure understanding and active participation.
  • Establish periodic evaluation cycles, incorporating community feedback to ensure ongoing improvement and adaptability.

Adopting responsible research assessment through open infrastructures:

  • Promotes genuine innovation, interdisciplinary collaboration, and societal impact.
  • Strengthens transparency, fairness, equity, and inclusivity in research evaluation processes.
  • Enhances institutional accountability and builds trust among researchers and stakeholders.

 Conclusion and Call to Action

Implementing responsible research assessment practices through open infrastructures and aligning with international reform initiatives is essential for fostering inclusive, fair, and impactful scholarly environments. Stakeholders must actively engage and commit to these transformative reforms to ensure ongoing excellence, integrity, and societal relevance in research.