External evaluation assesses the quality of the NCCR selection process

© © SNF 2021

A new evaluation study assesses the quality of the NCCR selection process and provides valuable information for its further development.

The Swiss National Science Foundation (SNSF) has commissioned the Nordic Institute for Studies in Innovation, Research and Education (NIFU) to conduct an evaluation of the selection process for the 5th call for National Centres of Competence in Research (NCCRs). The evaluation report that has now been published is based on a comprehensive analysis of documents from the application and review process, a survey of applicants to the fifth NCCR call, and interviews with reviewers, home institutions and others involved in the selection process. All in all, the report gives a positive assessment of the quality of the NCCR selection process and provides valuable information for its further development in the context of a future NCCR call.

The NCCRs investigate areas that are of strategic importance for Switzerland's research sector, the Swiss economy and Swiss society. An NCCR allows the funded teams to work on novel and complex research questions with a long-term perspective and to exchange ideas within a network. This enables universities hosting an NCCR as a home institution to establish or strengthen new research areas. Excellence and structure-building at universities are the central objectives of the NCCR scheme. The selection process for NCCRs, which takes place at intervals of five to six years, is very competitive. The SNSF is responsible for pre-selecting the most promising applications, applying primarily scientific criteria, but also taking into account the structural strengthening potential for the research area in question. The SNSF sends the selected NCCR applications to the Department of Economic Affairs, Education and Research (EAER). The latter makes the final decision on which NCCRs are to be funded.

The SNSF's preselection, which focuses on scientific quality, is of central importance for the results and impact of the NCCR funding scheme. For this reason, the SNSF decided to have the selection procedure evaluated by an independent institution. NIFU, a renowned research institute, was appointed as the evaluator because of its expertise in evaluating large funding programmes. In 2016, an initial evaluation study by NIFU on the third and fourth NCCR calls had provided specific recommendations on how to improve the NCCR selection process. Based on the recommendations, the SNSF made adjustments to the fifth selection process in 2017. The present evaluation was commissioned to examine the impact of these adjustments. In addition, the quality of the selection process was reexamined in view of subsequent NCCR calls.

The current evaluation focused on four elements:

  • Attractiveness and outreach of NCCR calls
  • Organisation of the procedure and competence of the experts
  • Independence, transparency, accountability, legitimacy and trust in the selection process
  • Effectiveness and efficiency of the procedure

The NIFU experts noted that hosting an NCCR is still very appealing and prestigious for applicants and their home institutions. They also believe that the SNSF selection process as a whole was well organised and went smoothly, notably due to the increased transparency and efficiency in the selection process. The majority of those involved in the process were satisfied, both with the care with which the NCCR applications were evaluated and with the SNSF's feedback on the applications. The adjustments made in the selection process on the basis of the first evaluation study had a positive effect, according to the report.

However, the NIFU report also identifies some aspects of the process that could be improved. These are the appeal of the NCCRs for the humanities and social sciences and for smaller universities, and the duration of the selection process. There is also room for improvement with regard to transparency and the rules for handling conflicts of interest. Both concern in particular the composition of the panels and the tasks of the panel members at the full proposal stage. The SNSF will address these critical points by defining specific rules on conflicts of interest for the NCCRs and ensuring that the process is as transparent as possible. In a position paper, the SNSF addresses these aspects and discusses further possible improvements that could be introduced ahead of the next NCCR selection process.

The 2016 and 2021 reports of the NIFU on the evaluation of the NCCR selection procedure, the SNSF's position paper on the 2021 evaluation, and the statements from the Sounding Board on the 2016 evaluation are published on the SNSF website.