Horizon 2020: Evaluation of Proposals

By Feb.05, 2014

This article outlines the overall procedure for Horizon 2020 proposals, which will be very similar to its predecessor the Framework Programme 7. However, given the different nature of Horizon 2020, including the more multidisciplinary nature of some projects and the more bottom up formulation of topics, there are a few differences in approach to note.

The general evaluation procedure in Horizon 2020 follows the same pattern as in FP7. Proposals are evaluated and scored by a number of experts who then have to find consensus on a final score for each proposal. Finally, proposals are ranked and funding is allocated according to the available budget and the ranking list.

The information below applies to the general Horizon 2020 evaluation procedure. Please note that there are variations for European Research Council and the Marie Skłodowska-Curie Actions.

Who manages the evaluation?

The evaluation of proposals is managed by the call co-ordinator from the Commission. The call co-ordinator is responsible for ensuring the smooth running of the evaluation process and takes certain decisions, including how many evaluators (beyond the minimum) are assigned to proposals.

Who are the evaluators?

There is a central database and an open call for expressions of interest for anyone who would like to become an evaluator. More information is provided in our article below.

How many evaluators will look at each proposal?

Initially, a minimum of two or three evaluators will examine each proposal remotely. For larger proposals there are likely to be more evaluators involved. A minimum of two evaluators have to be involved at the consensus meeting stage.

Up to how many proposals can be given to an evaluator?

There is no fixed rule on this and this is something the call co-ordinator ultimately decides. However, UKRO understands that on average a maximum of 10 smaller scale proposals would be given to experts, or around five large proposals.

Why will there be more remote consensus meetings for Horizon 2020?

At the consensus meeting, some or all evaluators who scored a proposal come together to agree a final score for each proposal. In FP7, the majority of consensus meetings were held in Brussels. With the larger volume of proposals expected and the new restrictions imposed by the eight month time to grant rule, it is likely that more consensus meetings will be held remotely. UKRO understands that the Commission is planning to use a special software that will allow online chat and or video to be used. Independent observers will still be part of this process but will also participate via chat or video in the case of a remote meeting.

Will the panel meetings also be remote?

After the consensus meetings, a panel meeting is organised to establish the ranking of proposals. A panel meeting is smaller than a consensus meeting and there are less experts present. These meetings will still be organised in Brussels and not held remotely.

What are the main differences between FP7 and Horizon 2020?

Expanding the database of experts: The Commission is trying to expand the database of experts significantly. Not just in numbers but especially in terms of expert profiles. This is to ensure that multidisciplinary or multisectorial proposals can be thoroughly evaluated.

More turnover: The plan is also to have a more robust turnover of experts. An expert can be active for up to four years in a row normally, and there is a limit on the number of days an individual evaluator can be used for. The Commission will have to ensure that there is sufficient rotation of experts.

Substantial changes and light negotiation: In Horizon 2020, experts are no longer required to make recommendations for substantial changes. This is supposed to result in a much shorter negotiation phase (“light negotiation”). While no definition of what qualifies as substantial is available, the overall idea is that instead of recommending that a new work package is added or changes are made to the structure of a consortium, the evaluator would now mark a proposal down if there are weaknesses from his or her perspective.

This would mean that only proposals that are more or less ready to start in their current form would enter the negotiation phase in the majority of cases. The negotiation would now predominantly concentrate on administrative issues and the part A of the proposal. This change is part of the Commission’s attempts to shorten the time to grant significantly.

Focus on impact: There will be a stronger focus on the Impact criterion for innovation actions and projects under the SME instrument, with the weighting being raised to 1.5 for these projects and impact being given priority over excellence (only for innovation actions and the SME instrument).

All applicants will have to outline clear strategies for dissemination and communication plans in their proposals. There is to date no specific new guidance on communication and dissemination for Horizon 2020 projects, but UKRO understands that the advice given in the FP7 guide remains valid in this case.

It is also expected that rules for situations when experts disagree will be made clearer and that conflict of interest rules will be tightened.