Research culture, open science and evaluations

Responsible evaluation of scientific research is one of the fundamental reform processes at both European and the global level. In addition to initiatives such as DORA and CoARA, the principles and reform directions are also coordinated within the framework of the European Union and the European Research Area (ERA).  

Slovenia is gradually making reforms to evaluation at several levels. At the national level, it is defined by the Scientific Research and Innovation Activities Act (ZZrID) and the Regulation on the Implementation of Scientific Research Work in Accordance with the Principles of Open Science. 

On the basis of ZZrID and the Regulation on the Implementation of Scientific Research Work in Accordance with the Principles of Open Science, ARIS is progressively implementing reforms in the evaluation of project proposals under competitive funding and in evaluations under stable funding. 

It is also developing more detailed regulations, specifications and guidelines in this area, such as: 

• General Act on the Stable Funding of Scientific Research Activity, 
• General Guidelines for the Institutional Self-Evaluation of Recipients of Stable Funding, 
• Proposed format for the self-evaluation report. 

You can read more in the Evaluations tab. 

Open science and the responsible evaluation of scientific research have an important influence on the development of a modern research culture. They promote greater openness, transparency, diversity and collaboration, and support high-quality, inclusive and socially relevant research. 

Approaches to evaluation 

The ways in which scientific research is evaluated have important implications for research practices. It is therefore necessary to introduce fair, transparent and effective approaches that directly value and recognise scientific excellence and societal impact, and take into account the diversity of research outputs, open research practices and research pathways. Thus, it is important that the evaluation takes into account the substantive achievements of the research work, rather than the place of publication, impact factor or other indicators. 

The role of ARIS 

ARIS systematically supports the strengthening of the knowledge, skills and supportive environments needed to promote open science and progressively reform the evaluation of research.  

ARIS works with research organisations and other stakeholders, with a particular focus on inclusive approaches and continuous learning. In this way, it contributes to a research system in which openness, transparency and accountable evaluation are part of the whole research cycle and knowledge is seen as a public good. In line with the Open Science Action Plan, ARIS manages the national coordination of members of the international CoARA (Coalition for Advancing Research Assessment), one of the most prominent international platforms for cooperation in the pursuit of responsible research assessment.  

Open access means free online access to scientific research outputs and the management of copyright through open licences.  

In addition to reading, storing and retrieving content, it enables users to use, distribute, transmit and publicly display it, and to produce and disseminate derivative works in a digital environment for responsible purposes. 

Open access to scientific publications

Open access to scientific publications means making the full text of a publication freely available on the publisher's website or repository. The author retains the material copyright, but the terms of use are set by the licence chosen, most often one of the Creative Commons licences. 

Useful links: 

• a list of open access journals: DOAJ, 
• a list of open access books: DOAB, 
• specifications for the publication of scientific articles in open access 2018–2023, 
• ARIS recommendations for the implementation of open science under the Horizon Europe guidelines. 

Research data are records of facts, such as numerical data and textual, audio and pictorial records, which are obtained by various methods and form the basis for scientific research. They are considered by the research community as an appropriate means of validating research findings.

Handling research data

Handling research data involves planning, collecting, storing, documenting, protecting, preserving in the long term and making data re-usable. This is done taking into account the FAIR principles: data must be findable, accessible, interoperable and reusable. 

A key tool for systematic research data management is a research data management plan, which defines how data will be managed throughout the research process. 

Evaluating scientific research is an important part of open science. Slovenia is gradually making reforms to evaluation at several levels. At the national level it is defined by the following: 

• Scientific Research and Innovation Activities Act, 
• Regulation on Conducting Scientific Research in Accordance with the Principles of Open Science. 

On this basis, ARIS is progressively implementing reforms: 

• in the evaluation of project proposals under competitive funding, 
• evaluations in the context of stable funding. 

ARIS is also working on more detailed specifications and guidelines, including: 

• General Guidelines for the Institutional Self-Evaluation of Recipients of Stable Funding, 
• Instructions for Assessment in External Institutional Evaluation, 
• Proposed format for the self-evaluation report. 

Reforms are also being pursued in cooperation with the international community, in particular in the context of ERA policies and Council of the EU conclusions on research evaluation and open science. 

You can read more in the Evaluations tab. 

Repositories are digital spaces that store digital objects, described by standardised metadata that makes them accessible. ARIS participates in and monitors the development of repositories at the national and international levels. 

For more information on the repository system, see Criteria for identifying trusted repositories and List of recommended repositories. 

Selected Slovenian subject repositories 

• ADP 
• CLARIN.SI 

Selected Slovenian institutional repositories 

• RUL 
• DKUM 
• RUP 
• RUNG 
• DiRROS 

General International Repository 

• Zenodo 

European and global cooperation is key to developing an infrastructure that supports open science. Major initiatives include: 

• ORE, the European platform for the open publication of EU publicly funded research results, 
• EOSC, the European initiative for a common digital space for accessing research data, publications, software and services. 

Major European and international initiatives: 

• Horizon Europe, 
• OpenAIRE, 
• Science Europe, 
• ERC projects, 
• CoARA. 

Slovenian legislative framework: 

• Resolution on the Scientific Research and Innovation Strategy of Slovenia 2030, 
• Scientific Research and Innovation Activities Act, 
• Regulation on the Implementation of Scientific Research Work in Accordance with the Principles of Open Science, 
• Open Science Action Plan for the implementation of Measure 6.2. 

In Slovenia, the permanent preservation of Slovenia's written cultural heritage in the fields of science, art and culture in digital form is made possible by the Digital Library of Slovenia, which is an important part of the modern infrastructure for educational and scientific research processes. 

• Open Access 
  Free online access to content on the publisher's website or in a repository, with copyright managed through a selected licence. 
• Diamond Open Access 
  A scientific publishing model in which journals and platforms charge neither authors nor readers. 
• Open Source 
  Software whose source code is publicly available and can be used, studied, modified and distributed by anyone. 
• Open data 
  Data that is accompanied by a licence setting out the conditions for its use, re-use and processing. 
• Research data management plan 
  A document that sets out how research data is collected, stored, documented, processed and published. 
• Repository 
  A digital repository for storing and making accessible digital objects described by standardised metadata.

Evaluation is a multi-step process that involves: 

• Evaluation of research programmes by field (ERC panels) 
  Independent experts assess scientific excellence and societal impact. 
• Institutional self-evaluation 
  The organisation carries out an internal audit and self-reflection. 
• External institutional evaluation 
  An independent expert team carries out a comprehensive assessment, including a visit to the organisation. 

All the parts are interconnected and form a single system. The results of evaluations provide an overall picture of how organisations are performing, include recommendations for improvement and lead to concrete actions. The allocation of stable funding can also be adjusted on the basis of these. 

Key criteria

Evaluations are based on three criteria:  

• scientific excellence, 
• societal impact, 
• the quality of the institutional research environment. 

In assessing the research environment, ARIS takes into account the organisation's governance, human resources policy and the development of young researchers, research infrastructure, collaboration and knowledge transfer, internal quality systems, and the implementation of open science, equal opportunities, ethics and integrity. 

• Scientific Research and Innovation Activities Act, 
• Regulation on the Implementation of Scientific Research Work in Accordance with the Principles of Open Science, 
• General Act on the Stable Funding of Scientific Research Activity. 

ARIS is also working on more detailed specifications and guidelines, including: 

• General Act on the Stable Funding of Scientific Research Activity,
• General Guidelines for the Institutional Self-Evaluation of Recipients of Stable Funding, 
• Proposed format for the self-evaluation report.