Posts tagged: Impact Case Study

Guidance on collecting impact evidence

If you are looking to generate impact from your research, please ensure that you engage from the start with the University Impact, Engagement and Environment Coordinator, Emma Sutton, and your School Impact Coordinator** so that the impact can be tracked and evidenced on an ongoing basis.

Key points to consider when you start a new research project:

  • What will be the indicators of impact? How will success be measured throughout and what needs to be captured?
  • Complete a stakeholder analysis to identify the potential beneficiaries of your work
  • Clearly demonstrate a pathway to impact: what steps will you take to engage with your stakeholders and how will you measure any benefits to them?
  • Articulate the significance and reach of the potential impact
  • Use existing and well-understood baselines and gold standards to measure your impact

Some examples of types of impact evidence that you could obtain:

  • Testimonials from organisations and individuals
  • Quotations from high-profile figures (obtained through interviews)
  • Participant feedback
  • Media mentions
  • Quantitative data (e.g. improved company sales, percentages demonstrating cost savings etc.)
  • Published reports as a result of research conducted
  • Guidelines/policy documents that cite your research

**Look to use both qualitative and quantitative data where possible!**

Points to remember:

o Ensure that information is robust and credible

o Ensure that information is independently verifiable

o Link evidence to clear targets and indicate whether these were met or exceeded

o Provide evidence of research being widely disseminated, e.g. through tweets, blogs, access to websites, press coverage, broadcastings, downloads, sales

o Find ways of communicating the research as it progresses to develop wider impact along the way (not just at the end)

o Conduct exit interviews with the business if ending relationship/researcher if leaving institution – evidence of impact must be captured before departure

o Be able to demonstrate that without the research, the impact would not have occurred: how has the research made the difference?

Remember: the earlier you begin collecting and collating your impact evidence, the easier it will be to create your own impact case study!

Further information on impact evidence collection can be found on the REF intranet at: www.salford.ac.uk/ref

**School Impact Coordinators are as follows: CSE – Prof Apostolos Antonacopoulos ELS – Prof Mike Wood / Prof Andy Miah H&S – Prof Neal Hazel Institute for Dementia – Dr Gemma Lace-Costigan SAM – Dr Pal Vik SBS – Prof Phil Scarf SOBE – Prof Peter Walker


Preparing for impact peer review

This month we are conducting an internal peer review of our potential impact case studies in preparation for REF2021. Twenty of these will subsequently be selected for external peer review in June.

Each Unit of Assessment (UoA) team has been asked to assign at least two reviewers to each impact case study (1 x Lead reviewer from the same UoA, 1 x Associate reviewer from a different UoA and, optionally, 1 x Non-academic reviewer [industry partner or similar]).

Reviewers have been given 4 weeks to look over the case studies and provide their feedback on a review sheet covering each of the 5 main areas in the impact case study (Summary of the impact; Underpinning research; References to the research; Details of the impact; Sources to corroborate the impact).

Each of the 5 aspects of the case study form are rated using a traffic lights system (red, amber, green) to indicate whether this is:

• an area requiring significant development (red)

• an area requiring some improvement (amber) or

• an area that is well developed and on track for submission (green).

An overall impact case study traffic lights rating is then provided at the bottom of the feedback sheet to indicate:

  1. evidence of reach and significance
  2. potential for submission of the case study to REF2021.

Feedback will be provided to the case study leads during May 2019.

This process will help inform decision-making within each UoA and will also identify where there is a need to focus resource for the final year of the REF process.

Go to www.salford.ac.uk/ref to check out some examples of annotated case studies from our 2018 external peer review.


Refining your impact case studies

Our potential case study leads for REF2021 have recently submitted the second draft of their impact case studies, which will be assessed as part of an internal peer review process in April. Twenty of these will then also be selected for external peer review in June.

At this stage in the process, feedback from colleagues can be key in ensuring that the case studies reach their full potential. Following the internal and external peer reviews we will have just over a year to generate some more impact, collect impact evidence and refine the narrative further before the REF submission deadline.

With this in mind, here are a few key points for our case study leads, or indeed anyone submitting an ‘impact statement’ for funding purposes or similar, to consider.

Key points to remember:

• Convincingly demonstrate the robustness and quality of the underpinning research in the first instance

• Distinguish between the underpinning research and resulting impact: establish the causation and make sure that there is a golden thread running through the narrative

• Do not focus too heavily on dissemination at the expense of resulting impacts: make sure you are not purely describing your pathway to impact

• Clearly articulate each of the impacts claimed, and their apparent significance and reach

• Ensure there is sufficient corroboration of the impact using appropriate evidence (testimonials, quotes from key stakeholders, citations in policy documents or in the media, documented changes to guidelines etc.)

• Where web pages are used, ensure you have preserved them (screen shots etc.) and don’t just use standard links that may become broken over time

Generating impact

If you are looking to generate some further impact to bolster your case study, don’t forget to consider the following:

1. Create a pathways to impact statement to clearly set out the impacts you are hoping to achieve

2. Complete a publics/stakeholder analysis to identify who you are hoping to influence

3. Engage with your stakeholders at every stage in the research process

4. Identify activities to engage with your publics

5. Drive impact online by developing a social media strategy

Go to www.salford.ac.uk/ref to check out some examples of annotated case studies from our 2018 external peer review.


Confirmed Impact Guidance for REF2021

With less than 2 years to go until our REF submission (deadline: Friday, 27th November 2020), the final set of guidance materials for REF2021 (including guidance on submissions and panel guidelines) was published on 31 January 2019 following wide consultation with the sector in late 2018. The final guidance documentation is available at www.ref.ac.uk/publications/

What does this mean for impact?

The salient points to take from the final REF guidance on impact case study submission are as follows:

Definition of impact for the REF

For the purposes of the REF, impact is defined as an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.

Impact includes, but is not limited to, an effect on, change or benefit to:

• the activity, attitude, awareness, behaviour, capacity, opportunity, performance, policy, practice, process or understanding

• of an audience, beneficiary, community, constituency, organisation or individuals

• in any geographic location whether locally, regionally, nationally or internationally

Impact also includes the reduction or prevention of harm, risk, cost or other negative effects.

Impacts will be assessed in terms of their ‘reach and significance’ regardless of the geographic location in which they occurred, whe

ther locally, regionally, nationally or internationally.

Impact also includes the reduction or prevention of harm, risk, cost or other negative effects.

Each of the four main panels (A, B, C, D) have slightly different requirements for the following:

• Continued case studies

• Indicators of quality for underpinning research

UoA Leads/Deputies are therefore encouraged to look closely at the panel guidance for their particular panel when reviewing impact case study drafts.

Submission requirements

• Each submission must include impact case studies (REF3 template) describing specific impacts that have occurred during the assessment period (1 August 2013 to 31 July 2020) that were underpinned by excellent research undertaken in the submitted unit. The underpinning research must have been produced by the submitting HEI during the period 1 January 2000 to 31 December 2020.

• This may include, for example, impacts at an early stage, or impacts that may have started prior to 1 August 2013 but continued into the period 1 August 2013 to 31 July 2020. Case studies will be assessed in terms of the reach and significance of the impact that occurred only during the period 1 August 2013 to 31 July 2020, and not in terms of any impact prior to this period or potential future or anticipated impact after this period.

• When writing case studies, submitting units should refer to the guidelines for presenting quantitative data set out in the ‘Guidelines for standardising quantitative indicators of impact within REF case studies’. These guidelines have been developed to enable more consistent presentation of quantitative evidence in case studies. This document (and a summary thereof) can be found separately at www.salford.ac.uk/ref under Impact Evidence Collection.

• More than one submitted unit (within the same HEI and/or in different HEIs) may include the same impact within their respective case studies, so long as each submitted unit produced excellent research that made a distinct and material contribution to the impact. In such cases, units may provide common descriptions of the impact arising, where they so wish.

• Impact case studies continued from examples submitted in 2014 will be eligible for submission in REF 2021 as long as they meet the 2021 eligibility criteria, including the length of the window for underpinning research and the assessment period for the impact described.

Go to www.salford.ac.uk/ref to check out all the REF guidance and more.


Research Impact Fund – Special Call

To support researchers at Salford in becoming more ‘impactful’, the University operates an internal Research Impact Fund, which offers:

• up to £1000 that must be match-funded by your School/Research Centre, to individuals and groups in support of activities that reflect the University’s desire to increase the impact and reach of its research, and/or highlight strategic engagement that builds upon the University’s vision to pioneer ‘exceptional industry partnerships’.

Or

• up to £1500 that must be match-funded by your School/Research Centre, to identified REF impact case study leads seeking to increase their dissemination and impact generation activities for the remainder of the current REF cycle.

Applications should be aligned to one or more of the following themes:

  • Strengthening interaction – seeking to nurture and build upon relationships with non-academic partners, aligning with strategic goal of the University – the Industry Collaboration Zones
  • Broadening research – in line with REF and funder requirements, to expand the reach and influence of research outcomes, in addition to introducing greater partner contribution into the design of future research
  • Promoting social benefit – demonstrating how the application of research-based knowledge might lead to practical and focused solutions at a range of scales

Suggested Activities                                                                                                 

The following list is not exhaustive:

  • The translation of research findings for non-academic audiences (e.g. policy reports, leaflets, audio-visual materials, etc.)                                 
  • Events or workshops with a focus on non-academic stakeholders (dissemination or activity-based)
  • The trialling of creative modes of public engagement (e.g. exhibitions or film screenings)
  • The commercialisation of research findings through IP protection and/or business engagement
  • The formation or strengthening of networks outside academia (e.g. visits/meetings to build relationships, or the initial development of a web/social media platform)
  • Exchange or placement activities (e.g. within an external non-HEI organisations, or through the placement of a non-HEI partner within the University)
  • The development of pedagogical materials (e.g. online training resources or ‘train the trainer’ sessions)

The 2018/19 Fund is currently open for a special call for new applications, with a deadline of Friday, 1st March 2019.

Further details and the application form can be found on the Impact Funding page at www.salford.ac.uk/ref

If you would like to find out more about the Research Impact Fund, or impact in general, please contact Emma Sutton, Impact, Engagement and Environment Coordinator on research-impact@salford.ac.uk


Peer Review of Impact Case Studies

According to Fast Track Impact’s calculations (see: http://www.fasttrackimpact.com/single-post/2017/02/01/How-much-was-an-impact-case-study-worth-in-the-UK-Research-Excellence-Framework for further details), the best impact submissions to REF2014, i.e. those achieving a 4* star narrative case study, had a currency exchange of some £324,000 (£46,300 per year between 2015/16-2021/22). By contrast, a 4* research output was typically valued at between £5,000-£25,000. Generally speaking, impact case studies are thought to be worth around 5 times more than outputs at higher full-time equivalents (FTEs).

As such, the huge potential value this may bring to institutions cannot be underestimated, particularly given the increased weighting of impact from 20% to 25% for the next REF exercise in 2021. Consequently, institutions employ a number of strategies and resources to ensure the best possible outcomes of their REF impact submissions. For example, there are reports of significant sums being spent by some universities in the REF2014 exercise on copy editors or science writers in order to create compelling narratives that would stand up to the scrutiny of the REF panel members.

A robust internal and external peer review process is one means of tracking progress over time in order to enhance and improve narratives and impact evidence ahead of the final REF submission in 2020.

 

Upcoming peer review events

The University of Salford is undertaking its first external peer review of draft impact case studies this Summer as part of its REF Readiness exercise. This will give the University a snapshot of where things stand and where improvements still need to made in the 2 years leading up to the REF submission. The feedback from the external peer review will inform the planned internal peer review due to take place in early 2019.

 

Dates for the diary include:

Monday, 18 June 2018 – Friday, 29 June 2018: External peer review of 10 x impact case studies across UoAs      This will include review and annotation of draft case studies, an overview report, notes on potential grades and advice on how to enhance impact.

Monday, 25 June 2018 – Friday, 20 July 2018: University of Salford Festival of Research       A month-long programme of events celebrating and promoting the University’s valuable research. This will include a REF-focused impact case study writing workshop, an impact ‘writing retreat’ and one-to-one mentoring on impact narratives.

Wednesday, 27 June 2018: Fast Track Impact case study writing workshop with Prof Mark Reed           Mark will focus specifically on the REF and what makes a good impact case study, how to improve your writing around impact, as well as evidence collection tips. This workshop will also include detailed external peer review of 4 draft impact case studies, with recommendations of how these can be enhanced and improved.

 

To book: https://myadvantage.salford.ac.uk/students/events/Detail/597642/staff-development-fast-track-t

 

Why not take this opportunity to look at the upcoming peer review meetings and events information on our REF Intranet site at: https://teamsite.salford.ac.uk/sites/sc02/REF2021/SitePages/Training.aspx