Tracking the Impact Beast

Impact, that mythical beast

When impact first burst onto the scene seven years ago it was so radical an idea, so controversial a concept, that we used to run ‘impact surgeries’, to which academics could bring their research ideas and where we would suggest possible ways in which their work could have impact. We were no impact experts, but had the objectivity and distance to see beyond the immediate research to its potential effect on the wider world.

In the intervening years the term impact has become more familiar, but still somewhat nebulous. There’s no longer a need for surgeries, but, like the Heffalump in Winnie the Pooh, people are still anxious about this mythical beast. They build it up in their imaginations to be much more than it is. They try to trap it and pin it down, but more often than not they end up following their own footprints. It’s viewed as either the Harbinger of the End of Academic Civilisation or, at the other extreme, the Salvation of Research Relevance. The truth, as ever, lies somewhere between these two.

In May, we held an event to try to map out this beast. We invited some of the best speakers on the subject to discuss it, from Jane Tinkler (London School of Economics and Political Science) and Stephen Curry (Imperial College London), to Andy Miah (University of Salford) and Steven Hill (Higher Education Funding Council for England). The latter’s thoughts were particularly intriguing, and he structured his talk around ‘six lessons’, which immediately brought to mind a festive religious celebration. This was apt: there was a slight sense of him providing us with quasi-mystical insights from a higher being. He was David Sweeney’s representative on Earth.

Inevitably, for Hill the inclusion of impact within the REF has been a positive one. He said he believes that it has given universities an opportunity to think objectively about what they do, and how they should act strategically. Hill quoted one anonymous university source who said, “In short, we know more about ourselves as a consequence of making a REF submission.”

Following the exercise, HEFCE surveyed the authors of impact case studies and narratives, and the most frequently mentioned benefit was the opportunity it gave to identify and understand the impact of research. Authors also welcomed the chance to reflect on strategy, for their own research or that of the wider faculty or university, as well as having a channel through which to promote and recognise the work of colleagues and their relationships with research users.

More substantially, formally recognising impact has led to structural changes. Evidence of impact is now more systematically captured, and impact activity is generally included as part of promotion applications and grant proposals. Hill went on to suggest that the assessment process itself has been beneficial. It has brought together academics and users, and the subsequent calibration and resulting judgements had given us confidence in this fledgling science. However, he did recognise that some issues remained: the templates (or narratives) were difficult to compare, and the lack of a requirement for evidence meant that much rested on the quality of the writing.

Furthermore, he said the exercise had proved how diverse impact could be. Not surprisingly different types of impact prevailed in different disciplines, from public health and disease prevention in the medical sciences, to informing government policy in the social sciences, to interacting with museums in the humanities. Going further, it was clear that the most effective and far-reaching impact resulted from the integration of knowledge across disciplines.

The REF had also punctured the myth that impact stemmed from peripheral or low quality research, Hill said. HEFCE had mapped out a scatter graph, with the grade point average of outputs on the X axis and of impact on the Y. The majority of the points were clustered in the top right-hand corner, suggesting that there was a correlation between quality in both axes.

Hill’s final point focused on the creation of the REF Impact Case Studies Database. The systematic collection of impact data has generated what he sees as an important national asset, and provided new insight into the relationship between research and impact. As a result the Heffalump is, perhaps, a little less nebulous; or at least it’s footprints are much, much easier to make out in the snow.

Hopefully by the time that REF 2020 rolls around we might have the hard evidence the proves the existence of the real thing.

This article first appeared in Funding Insight in August 2015 and is reproduced with kind permission of Research Professional. For more articles like this, visit www.researchprofessional.com