The Breathless Pursuit of Excellence

'Excellence of physique': soon to be a REF category
(Image: Wellcome CC BY)

The rhetoric of excellence has become all pervasive in higher education and beyond. We all aspire to excellence, but what does it actually mean? It's become an empty phrase that can mean anything to anyone - and that's its appeal.

Tracking the Impact Beast

Impact, that mythical beast

When impact first burst onto the scene seven years ago it was so radical an idea, so controversial a concept, that we used to run ‘impact surgeries’, to which academics could bring their research ideas and where we would suggest possible ways in which their work could have impact. We were no impact experts, but had the objectivity and distance to see beyond the immediate research to its potential effect on the wider world.

'The Sower of Discord'

Somewhere in Turkmenistan

Last week David Willetts suggested that the Government was considering 'franchising' the REF to countries overseas. Now read on.

A terrorist arms bazaar, somewhere in Afghan/Turkmenistan border, a la Tomorrow Never Dies. Amidst all the hardware is a small stall piled high with A4 brochures. A bald man in rimless glasses stands behind the stall. He's wearing a Life of Brian style beard.

Willetts (for it is he): Pssst!
Terrorist (points to himself): Who, me?
Willetts: Yes! Are you interested in causing (looks to left and right) chaos and confusion?
Terrorist: Well...
Willetts: Do you want to blow apart civilisation? Destroy social conventions? Do you want to strike  at the heart of the intelligensia?
Terrorist (suspiciously): Maybe...
Willetts: Ah! Come! I have a wondrous weapon! More powerful than any nuclear warhead! More dangerous than a box of hand grenades without their pins!
Terrorist (looking at the stall, and picking up a brochure entitled 'Panel Criteria and Working Methods'): But this is just a...
Willetts: (waving his arms) Don't be fooled my friend! This has the potential to return society to a prehistoric struggle for survival! (Grabs the brochure and stabs his fingers at its cover) This has the power to make 'civilised' people claw each other's eyes out, and trample each other into the dirt! Not only that, but you can force the smartest people in your country to spend hours - weeks! - years! in futile bureaucracy!
Terrorist (shaking his head): That will never happen.
Willetts: It will! I have tried it and I know it works! You don't need guns, you don't need tanks. All you need is 'The Sower of Discord'!
Terrorist: 'Sower of Discord'? Is that its name?
Willetts: Yes! It is the Armageddon Machine! But we must move stealthily. In my country it has been codenamed (whispers): 'The Research Excellence Framework'.
Terrorist: So how does it work, this Sower of Discord?
Willetts: Ah! That is the clever part! This is the - mwhahahah! - genius of my plan! These intelligent infidels are the creators of their own destruction! See, I give them a pile of money and say that they have divide it up amongst themselves. But they mustn't do it equitably! Oh no! They must work out who is the 'cleverest' amongst them.
Terrorist: That doesn't sound too hard.
Willetts: Oh but it is! How do you measure 'clever', eh? Eh?
Terrorist: Hmm.
Willetts: Yes! They tear themselves apart with H Indexes, and impact factors, and all sorts of crazy bibliometrics! But my friend, I've not got to the best bit. Not only do they have to work out who is the cleverest, but also who is the most - heh! heh! -  'impactful'.
Terrorist: What does that mean?
Willetts (eyes gleaming): Ah - ha! That is the genius! No one knows! And we do not tell them! We just say, 'work it out for yourselves!' Hahahahahahahahahahahahaha! It creates institutional and personal madness! Madness!
Terrorist (with disgust): But that is...monstrous. What kind of sick individual are you? I may be a terrorist, but I do have some standards. You keep your, your...so-called 'Research Framework Excellence'. I'm off to buy some canisters of sarin gas.

...Or Not...

At the same time as the REF Consultation meeting was producing quite an upbeat view of Impact (below), the Centre for Business Research has said that too much pressure to achieve “impact” from research may hold back the work of universities. 'Such pressure could undermine some of the core strengths of many universities in particular if it leads to less basic research,' said the report.

The research is based on a web-based survey of more than 22,000 UK academics active in research or teaching in 2008-09. The full report can be seen here.

Academics Welcome Impact Shock!

A group of around 50 academics have given a surprise endorsement to the REF Impact plans. At a HEFCE Consultation event on the REF proposals in London yesterday feedback from the 'breakout' groups suggested that academics now accept the need to justify the impact of their research to the government. David Sweeney, who opened the event, highlighted the massive increase in QR funding since 2002/03, up some 41%.
Whilst academic freedom was ‘a gift’, one that should be nurtured and cherished, and institutional autonomy was important, universities ‘shouldn’t be monasteries.’

He went on to explain that HEFCE was now fully behind peer review, a claim that couldn’t have been made a few years ago, and the battle they had had in establishing this should not be taken for granted.
HEFCE was aware of the ‘challenges to assessment’, but he believed that many of these would ‘meltaway’ when faced by ‘very bright people’ tackling them. He emphasised that the REF was not about quantifying impact, focussing on economic impact, assessing impact of every researcher or output,trying to predict future impact, discouraging curiosity-driven research, or trading off impact against excellence. 'High impact that results from poor research is not a feature of this exercise,’ he said.

Full notes on the event are available if you contact me.

Nobel Prize Winners Sign UCU Anti-Impact Petition

Nobel Prize winners Venki Ramakrishnan, Tim Hunt, Brian Josephson, John Walker, Richard Roberts and Harry Kroto have signed a UCU petition calling on the government to rethink its plans for assessing Impact as part of the REF. Over 3,000 others have also signed it, including Richard Dawkins and Robert Winston.

The UCU argues that 'the REF proposals are founded on a lack of understanding of how knowledge advances. It is often difficult to predict which research will create the greatest practical impact. History shows us that in many instances it is curiosity-driven research that has led to major scientific and cultural advances.'

The petition calls on 'the UK funding councils to withdraw the current REF proposals and to work with academics and researchers on creating a funding regime which supports and fosters basic research in our universities and colleges rather than discourages it.'

Russell Group: Give us the Cash

In a news item that must match 'The Sun Rose this Morning' for it's shock value, the Chair of the Russell Group, and VC of a Russell Group university, said that the REF should ensure that QR funding is concentrated on Russell Group universities. Professor Michael Arthur used a HEPI conference last week to state his view. Criticising the outcome of RAE2008, which funded excellent research wherever it was found, he said that "if we carry on with that trend ... and take money away from those universities that have been highly successful in the past, we end up with a progression to mediocrity."

He went on: "How many well-funded research universities do we need? I don't believe it is 169. I'd like to suggest it is somewhere between 25 and 30." How many universities are there in the Russell Group? 20. We wait to see who the lucky 5-10 outsiders will be who are invited to share King Arthur's feast.

Ref Impact - 'Too Much Weighting'

The Higher Education Policy Institute (Hepi) has issued a critique of the Ref. Although largely welcoming the proposals in the Consultation document, it calls for a rethink of the Impact requirement. In their press release, Hepi say that 'it seems unwise to attribute so great a weight to a feature that is in effect experimental'. Comparing it to the cultural changes following the 1992 assessment, Hepi's Director Bahram Bekhradnia said that 'this proposal needs to be handled extremely carefully...the proposed impact requirement will influence behaviour in ways that can only be speculated about at present.'

The Evolution of Impact

A packed Senate Chamber debated two different perspectives on Impact on Thursday. Prof Peter Taylor-Gooby gave a 'forensic' analysis of the Impact proposals contained in the Ref, and highlighted the uncertainty that still hung over them. Dr Steven Hill gave the view from RCUK, the political impetus for the policy, and the current thinking on how individual Research Councils dealt with Impact.
Both talks - whether intentionally or not - demonstrated that the assessment of Impact was still evolving, and all players were still feeling their way. For the Ref, there was the question of creating consistency, particularly given the introduction of 'users' to the panels, and the relatively tight timeframe Hefce had for benchmarking. For RCUK, there were mixed messages coming out of Polaris House: on the one hand Impact was considered to be very important; on the other it was only a secondary criterion in the assessment process. Elsewhere they appear to encourage global Impact, although their government paymasters might wish to prioritise UK Impact.
Notes from the Impact Seminar will be posted on the Research Services website shortly, or contact me for an electronic copy.

Monday 12 October: Meeting to Discuss Ref Consultation

A reminder that the Directors of Research Network Meeting will be open to all staff on Monday. The issue to be discussed is the Research Excellence Framework Consultation Paper. It will take place in Keynes Seminar Room 14 between 10am -12pm. All welcome, but do let me know that you intend to come.

Tories: Assessment of Impact 'Clunky'

David Willetts, the Conservative shadow education secretary, has queried the assessment of impact in the Research Excellence Framework (Ref), according to ResearchResearch.
At a fringe meeting at the Tory conference, Willetts criticised the "clunky" assessment of impact now required by the research councils. Its saving grace, he said, was that leaders of the research councils had told him that they didn't let it affect their grant-making decisions. And he went on to warn: "We don't want to import that into the Ref."
The statement sharpens the differences between Conservative and Labour over the future of the Ref. While Willetts has shifted the Conservatives away from the new approach outlined in the consultation published by HEFCE last month, vice-chancellors are saying that the impact agenda has come directly from Alistair Darling's Treasury.
Willetts also said at the meeting that he thought the biggest challenge for universities was in teaching, not research, and gave a warm endorsement to the importance of research in the social sciences and humanities.

Mammoth Trap - Part Two

Following on from David Mitchell's piece on the Ref in last week's Observer (see this post), Prof Steve Fuller of Warwick is suggesting that the comedian become a board member of either the ESRC or AHRC. He makes his case in his blog, calling Mitchell 'one of the most articulate and insightful commentators on the state of higher education today.' He goes on, 'what he brings is the right sensibility – or at least a sensibility that isn’t more forcefully voiced even by academics themselves. It also highlights something that at least academics easily forget – that people who work in the arts and the media, the source of an increasing portion of the nation’s wealth, are among the biggest boosters of academia for what we would regard as purely academic reasons.'

'Open' Directors of Research Network Meeting: Ref Consultation

There will be a Directors of Research Network Meeting open to all staff on Monday 12 October 2009 at 10am, to discuss the proposals contained in the Ref Consultation paper (see this post). All welcome; venue to be confirmed.

Mammoth Trap

'The Research Excellence Framework is starting to ask what sorts of curiosity our culture can afford, and that scares me even more than the demise of the silly survey because it strikes at the heart of what it means to be civilised, to have instincts other than survival. If academic endeavour had always been vetted in advance for practicality, we wouldn't have the aeroplane or the iPhone, just a better mammoth trap.'
More from David Mitchell in the Observer on the Ref here. Thanks to Gill for highlighting this.

Brace for Impact

‘Social and Economic Impact’ has become a key factor in funding research. Applicants to the Research Councils now have to explain the wider impact of their research, and Hefce has announced its intention to consider impact as part of the forthcoming Ref (see post, below).
However, this does not mean that you should change the research you do. Instead, you should think about how your research affects the wider world. So, when working on Research Council applications, bear in mind four points:

Broad definition
Think broadly about the potential effects of your research: ‘impact’ can include cultural, economic, health, social, environmental, and legal effects of your research.

‘Stories’
Think about your research from a more general perspective: what the Councils are really looking for are potential ‘case studies’ or ‘stories’ that can be used by them to justify their funding to government and the public.

Long term
Think about the long term impact of your research: The Research Councils recognise that it sometimes takes a long time for a project to have impact (an MRC/Wellcome study suggested that the average time lag is 17 years).

‘Due diligence’
Finally, what the Councils really want is reassurance that you have applied ‘due diligence’ to considering the ‘Impact’ of your research. Your Impact Plan should demonstrate this.

Ref Consultation Document Published

Hefce have published a consultation paper on the successor to the RAE, the Research Excellence Framework (Ref). ‘Impact’ is going to play a large part in the assessment, and the number of units of assessment have been drastically cut. A summary of the proposals is reported in Research Fortnight , and the full consultation can be seen here.