Horizon Scan 2021

What does the future hold?
Close your eyes and hold on tight...
Photo by Waldemar Brandt on Unsplash

Each year Research Professional invites me to look to the year ahead and try and predict what's on the cards in the world of research funding. Inevitably I get it wrong. But will 2021 be the year I'm spot on? Well, it's only January and I've already missed an insurrection in America and a second impeachment of the President. My crystal ball is as hazy as ever...

What You Need to Know: How other Countries Assess Research

Two ends of the South African research assessment scale: kak and lekker.
Photo by Greg Bakker on Unsplash

In the UK the REF has become an unavoidable feature of the research landscape. But how do other countries assess research? We look at one example - South Africa - and their very different solution to the same challenge.

HEFCE Announces 'Confidence and Supply' Logo

We were shocked to learn today that, despite all the excellent work by the Fundermentals Creative Team, it's logo was not chosen to represent REF2021.

Logo: not chosen

Research Funding and the Premiership

It is a truth universally acknowledged that a university in possession of a good research income must be in want of more. At the Association of Research Managers and Administrators conference in Birmingham earlier this year, Randolph Haggerty of the University of Leeds and Ann Charlett-Day of the University of Sheffield explored this perplexing maxim. Can we really all grow bigger? And, if so, how?

Like Oliver Twist, all universities want more, and many are met with disbelief. Haggerty came clean and said that his own institution aimed to increase its research income by 50 per cent. Others are even more ambitious: Kent aims to double its income and the University of Huddersfield aims to triple it. This, Haggerty suggested drily, was “challenging”.

Nevertheless, the present climate gives universities little choice. The Research Excellence Framework uses research income as a metric to measure a healthy research environment. The results of that feed through into league tables, which, in turn, inform the recruitment of students and staff.

Schrodinger's REF

Schrodinger's REF: Portable and Non-Portable

Yesterday David Sweeney unveiled his initial plans for REF2021. ‘We’ve responded fully to the recommendations of the Stern Review, together with the comments and feedback we received in the consultation that followed,' he said.

‘Although they were diametrically opposed to each other, I’m delighted to say we’ve managed to create a hybrid framework for the next exercise which manages to accommodate all views.  

‘As a starting point we used the excellent 4* work of Erwin Schrödinger. To meet the conflicting views of all stakeholders, we have devised a system whereby everyone will be submitted, but at the same time there’s the option whereby no one will be. The system will be completely inclusive but absolutely exclusive. It will be a 100% submission and a 0% submission. I believe these two states of being can fully coexist.

'Citations Are Peer Review by the World'

Prof John Mingers

The research excellence framework - or REF - has become such an established part of the higher education landscape that it’s hard to imagine a world without it. Since 1985 it has provided the navigation points in a turbulent and changing landscape, marking progress and, like a modern day Lachesis, measuring the threads of individual research in readiness for Atropos’ shears.

As we gear up for REF2021 and wait to hear from HEFCE about what form it will take, it seems an appropriate time to take stock and question whether the game is worth the candle, and if the measurement of research should be undertaken differently.

Impacting Parliament: Giving MPs the Information They Need

 In the second of her guest posts, the University's Impact and Engagement Officer, Maddy Bell, shares some notes on what academics should bear in mind when working with Parliament.

Parliament is actively seeking external input to make informed decisions based on sound evidence. But MPs are under huge pressure – working long hours with limited opportunity to absorb key information. Given this, academics can play a crucial role in providing them with the knowledge and understanding they need to make better policy. 

At the end of last year I attended an event run by the Houses of Parliament Universities Programme that outlined the ways in which academics can engage.  Here I summarise these, and the potential impact your research can have from working with Parliament.

How to Write a 4* Article

Prof Mark Reed

Last week Prof Mark Reed, Professor of Socio-Technical Innovation at Newcastle University and the man behind Fast Track Impact, tweeted some thoughts on how to write a 4* paper for the REF. Here he explains his thinking in more detail.

_____________

How do you write a 4* paper for the Research Excellence Framework (REF)? It is a question I’ve asked myself with some urgency since the Stern Review shredded my REF submission by not allowing me to bring my papers with me this year to my new position at Newcastle University.

Obviously the answer is going to differ depending on your discipline, but I think there are a few simple things that everyone can do to maximize their chances of getting a top graded research output.

Tracking the Impact Beast

Impact, that mythical beast

When impact first burst onto the scene seven years ago it was so radical an idea, so controversial a concept, that we used to run ‘impact surgeries’, to which academics could bring their research ideas and where we would suggest possible ways in which their work could have impact. We were no impact experts, but had the objectivity and distance to see beyond the immediate research to its potential effect on the wider world.

Government Follows up REF and TEF with AEF and FTEF

Jo 'Edmund' Johnson: Sleep Excellence Framework 'to follow'

Following news of the new Teaching Excellence Framework (TEF), the Government has announced that it intends to introduce an Administration Excellence Framework (AEF) and a Free Time Excellence Framework (FTEF).

'Research and Teaching are only a part of an academic's life' explained Jo 'Edmund' Johnson, Minister of State for Universities, Science and Excessive Assessment. 'We want to get a full understanding, and star rating, for absolutely every single part of it: every moment of every day.'

Impact: What Works?

The Award-Winning
Julie Bayley

Whilst impact has become an accepted fixture on the research landscape since it was introduced by the Research Councils in 2008, there is still a nervousness about what it is and how it should be recorded.

Fortunately help is at hand: Julie Bayley has developed a national reputation in supporting colleagues in understanding impact at Coventry University, and she accepted our invitation to come to Kent and talk about her experiences.

Exclusive: Design of REF2020 Logo Revealed

With just fifty months until the likely submission date for the next REF, HEFCE has announced the design for the REF2020 logo.

The design was revealed at a showbiz ceremony in the heart of Bristol's glittering Abbey Wood. An expectant crowd of literally tens - or ten - of Higher Education's finest celebrities gathered to drink mulled cava and eat slightly stale Hula Hoops and listen to the announcement.

'The Sower of Discord'

Somewhere in Turkmenistan

Last week David Willetts suggested that the Government was considering 'franchising' the REF to countries overseas. Now read on.

A terrorist arms bazaar, somewhere in Afghan/Turkmenistan border, a la Tomorrow Never Dies. Amidst all the hardware is a small stall piled high with A4 brochures. A bald man in rimless glasses stands behind the stall. He's wearing a Life of Brian style beard.

Willetts (for it is he): Pssst!
Terrorist (points to himself): Who, me?
Willetts: Yes! Are you interested in causing (looks to left and right) chaos and confusion?
Terrorist: Well...
Willetts: Do you want to blow apart civilisation? Destroy social conventions? Do you want to strike  at the heart of the intelligensia?
Terrorist (suspiciously): Maybe...
Willetts: Ah! Come! I have a wondrous weapon! More powerful than any nuclear warhead! More dangerous than a box of hand grenades without their pins!
Terrorist (looking at the stall, and picking up a brochure entitled 'Panel Criteria and Working Methods'): But this is just a...
Willetts: (waving his arms) Don't be fooled my friend! This has the potential to return society to a prehistoric struggle for survival! (Grabs the brochure and stabs his fingers at its cover) This has the power to make 'civilised' people claw each other's eyes out, and trample each other into the dirt! Not only that, but you can force the smartest people in your country to spend hours - weeks! - years! in futile bureaucracy!
Terrorist (shaking his head): That will never happen.
Willetts: It will! I have tried it and I know it works! You don't need guns, you don't need tanks. All you need is 'The Sower of Discord'!
Terrorist: 'Sower of Discord'? Is that its name?
Willetts: Yes! It is the Armageddon Machine! But we must move stealthily. In my country it has been codenamed (whispers): 'The Research Excellence Framework'.
Terrorist: So how does it work, this Sower of Discord?
Willetts: Ah! That is the clever part! This is the - mwhahahah! - genius of my plan! These intelligent infidels are the creators of their own destruction! See, I give them a pile of money and say that they have divide it up amongst themselves. But they mustn't do it equitably! Oh no! They must work out who is the 'cleverest' amongst them.
Terrorist: That doesn't sound too hard.
Willetts: Oh but it is! How do you measure 'clever', eh? Eh?
Terrorist: Hmm.
Willetts: Yes! They tear themselves apart with H Indexes, and impact factors, and all sorts of crazy bibliometrics! But my friend, I've not got to the best bit. Not only do they have to work out who is the cleverest, but also who is the most - heh! heh! -  'impactful'.
Terrorist: What does that mean?
Willetts (eyes gleaming): Ah - ha! That is the genius! No one knows! And we do not tell them! We just say, 'work it out for yourselves!' Hahahahahahahahahahahahaha! It creates institutional and personal madness! Madness!
Terrorist (with disgust): But that is...monstrous. What kind of sick individual are you? I may be a terrorist, but I do have some standards. You keep your, your...so-called 'Research Framework Excellence'. I'm off to buy some canisters of sarin gas.

Schizophrenia and the Funding of Regional Universities

Hull: one of 'Wave 2 Growth Hubs'

In February the Government announced that it was offering £32m to 20 regional cities in the UK to support economic growth and development. The cities, which included areas as diverse as Milton Keynes, Hull, Stoke and Sunderland, were offered the funding in return for a promise to 'improve a serious local economic problem'.

Hail the Phoenix

Joni Mitchell captured it perfectly: 'you don't know what you've got til it's gone'. The repercussions of the demise of the 1994 Group will, I think, be felt for some time yet. Whilst I took plenty of sideswipes at the 1994 Group in its last, doddery years - it was such an easy target, wasn't it? - the gap that it's left is a significant one for the UK's higher education sector.

In essence, we've now got a one brand market place. The Russell Group is the Acme Corporation. It is GUM. I've heard rumours that the Russell Group was not exactly passive in the self destruction of the 94 Group last summer. If this is true, it should be ashamed of the part in played. Whilst all is fair in love, war and the feuding of mission groups, having one citadel of research is not good for the long term health of the UK's research ecosystem.

I've made no bones about believing that increasing funding concentration is a Bad Thing, and  I hope that HEFCE sticks to its guns and 'fund excellent research in all its forms wherever it is found'. I think it would be incredibly detrimental to the whole sector if we moved to a situation where we had a divide between 'research' and 'teaching' universities. It would be harmful to the early careers of many academics, but also to the experience of students. Research should inform teaching, and teaching inform research. Having a strong research ecosystem across the sector is good for the UK as a whole.

However, to ensure its survival this ecosystem needs to be protected. Someone needs to fight the corner of the non-RG universities. Someone needs to speak for the majority. I think its disingenuous - or just plain stupid - to think that individual universities can speak as loudly or be heard as widely as a collective voice.  In the weeks that have followed the closure of the 1994 Group a number of possible candidates have been suggested, such as Universities UK (UUK), University Alliance, and even the Association of Research Managers and Administrators (ARMA). 

I don't think any of these are appropriate. Both UUK and ARMA should be looking after the interests of all in the sector, both non-RG and RG. The University Alliance has a particular mission in supporting and encouraging innovation within it member institutions. What we need instead is either to get rid of all mission groups and just have a single body speaking on behalf of the whole UK HE sector, or a body who can safeguard the smaller research intensive universities. The former is never going to happen; the RG brand has become too useful to its members. The latter isn't going to happen anytime soon either: too many fingers have been burnt, I think, in the sad end of the 1994 Group. The idea behind the Senate Group was a good one, a positive one, but it was ultimately scuppered by delays and self interest.

What we are left with, for the time being, is the RG, and a patchwork of regional groupings, from N8, M5, GW4 and Kent's own Eastern ARC. These do, I think, offer a solution for the moment. They allow the member universities to be more than the sum of their parts, and to spark interesting, interdisciplinary collaborations. They may even, in time, lead to a more 'federalised' sector. But there's always a danger that 'federalised' might lead to 'factionalised', with inter-regional scrapping and the building of walls between the groups.

Despite what some say, and despite the silly playground politics they engender, I think 'mission groups' do have their place. Or rather, in a world where some belong to a powerful mission group, the rest need to have the advocacy offered by one. Let's hope that a phoenix rises from the ashes of the 1994 Group. And soon.

The Difficulty of Impact

Definitely not David Sweeney

The LSE'Future of Impact' conference yesterday had a valedictory feel. It was the last huzzah for ‘The Impact of Social Sciences’, a HEFCE-funded project which had sought to 'develop precise methods for measuring and evaluating the impact of research in the public sphere.' 

Whilst they have achieved a lot – their blog is a fantastic forum for discussing issues relating to impact, for example – I think even their fiercest admirers would be hard pressed to say they had succeeded in this.

That’s not necessary their fault. Rather, it's the inevitable consquence of grapling with the amorphous, shape-shifting beast that is impact. This was particularly apparent in the third session yesterday, ‘Next Steps in Assessing Impact,’ which saw the three speakers almost come to blows over what impact is, who wants it, why, and how fast. This was partly down to the ever-entertaining David Sweeney, the architect of the REF. Like an embarrassing uncle at a wedding, he can always be relied upon to speak his mind. Loudly.

He took issue with the previous speaker, Julia Lane, who had been talking about StarMetrics. I’ve spoken about this before on this blog. To me, it sounds like an eminently sensible system (although I don’t think she did it justice here); Sweeney, however, begged to differ. Not only did he think that academia should not snap to attention when governments ask for data, but he implied that the system was only partially successful at collecting the right information. 

Similarly, he turned on the third speaker, Cameron Neylon. Neylon had suggested that Twitter could be used to monitor and engage with users of academic research. He gave an example of South African research which had been retweeted by someone working in community health promotion. ‘That’s not impact’, said Sweeney, dismissively.

And therein lies the problem. Here were three eminent speakers working at the coal face of impact. And yet, between them, they couldn’t reach agreement on what constituted impact, or why we should be doing it. If they have problems defining impact, what hope is there for the rest of us?  

Private Eye Lookalikes: 4* Output?

Having cut my teeth (metaphorically) on the great and the good from RCUK (for example here, here and here) I have finally reached the acme of the lookalike world. Never mind an article in Nature, there is no more prestigious publication in the lookalike discipline than Private Eye.

My life is complete. However, it does beg the question: is this a REFable output? If so, what is the impact factor of Private Eye? Is there an H index for lookalikes? How can I track my citations? Do 'likes' on Facebook count? *Sigh* Having reached this acme, I realise that all I've done is open up a world of academic angst...

The REF and Edited Books

There was an interesting piece by Prof Andy Miah on the Guardian Higher Education Network blog last week, which looked at what academics should be doing at this late stage to bolster their outputs for the REF.

Prof Miah started by recognising two points: firstly, time is short, so there's a limit to what you can still publish; and secondly, there is wide variance between disciplines as to what is regarded as prestigious or good. Journal articles are regarded much more favourably in the STEM subjects, for instance, than they are in the Humanities, where there is more of a tradition of monographs.

The focus of his piece, however, was edited books, which he described as 'the biggest loser' in RAE2008, when very few were submitted. This was because editing does not equate to research, but also because there was some ambiguity over the peer review of book chapters, and the difficulty in getting book chapters cited. But these volumes are not without worth: they offer a useful way for early career researchers (ECRs) to be published, as the process is often more flexible and supportive.

Will this mean, therefore, that there is a certain amount of discrimination against ECRs in REF2014? On balance I think not: whilst this is the implication of Miah's argument, most ECRs worth their salt will have been able to publish up to 3 outputs within the time, without having to factor in a book chapter. From the preparatory REF meetings I've been part of it's certainly not been to big an issue. However, it is worth highlighting it: the more entrenched these assessment processes become, the more they will affect behaviour, and the more likely that a time will come when edited volumes are a thing of the past.

That's fine if that's what everyone wants, but we shouldn't stumble into it blindly. If the academic community and the publishing industry see the worth of edited books then they need to make sure that some allowance is made for them in the REF. Otherwise academics will continue to prioritise journals and monographs, and edited books will wither on the vine.

Final REF Criteria Published

The final versions of the REF Panel Criteria and Working Methods were published yesterday, and are available here.

Highlights include:

• outputs pre-published in 2007 may be submitted as long as they were not submitted to RAE2008
• the number of outputs required of ECRs remains as in the draft criteria
• a new tariff has been introduced for the number of outputs required of staff who work part-time, go on secondment or have career breaks
• working methods are now common across the whole exercise; not distinguished by panel
• a reduction of one output per period of statutory maternity or adoption leave is allowed, with no minimum qualifying period
• a reduction of one output per period of ‘additional paternity or adoption leave’ lasting 4 months or more.

Thanks to my colleague Clair Thrower for looking at the criteria and picking out these points. If you have specific questions about this do drop her a line.

Autumn Newsletter Out Now

Right! Time to stop whatever you're doing, and make for the newstands: the autumn edition of our newsletter, ResearchActive, is out.

It's a bumper edition this term, and includes:

• details of the shiny new Peer Review system that was launched at the beginning of October;
• a snapshot of selection of recent awards;
• information about the new Grants Factory programme;
• all the funding gossip from Brussels and Swindon;
• an overview of the research interests of the 25 new academics who have joined the University recently;
• pass notes on the REF;
• contact details for everyone in Research Services.

So check your pigeon holes, and drop me a line if that familiar yellow pamphlet isn't there; I'll send you a pdf of it by return.