Building an Internal Peer Review System

ARMA training: The Royal York Hotel

I took part in an ARMA event today. The focus was on how we, as research administrators and managers, could help to improve the quality of applications. I talked about the Grants Factory programme and Kent's internal peer review system. I shared the platform with Linsey Dickson from Heriot-Watt and Sue Coleman from Edinburgh, and it was interesting to hear what they did at their institutions, and how they compared and contrasted.

However, for me the most interesting part of the day was when the delegates talked amogst themselves about what an ideal peer review system would look like, and what challenges they would face in introducing one.  Common themes emerged:

• Timeliness is key. Applicants need feedback as early as possible. Of course, this isn't always possible: funder deadlines might be too tight, or applicants' collaborators may not give access to proposals early enough. Or (whisper it) the applicants themselves might just do things last minute.
• Feedback has to be useful. Well, durr. Perhaps I need to rephrase: reviewers have to be forced to give feedback which can be used. It's not enough to say, 'fine', or 'needs more work.' What applicants need is detailed feedback: what needs changing, and how?
• There needs to be more than one reviewer. This is something that Kent's system includes (our proposals have to be seen by a disciplinary reviewer and a funder reviewer), but some of the suggestions around the room included a user reviewer, or applicants having the opportunity to nominate their own, but having no guarantee that they'd be used. Additionally, reviewers should be compensated for their time, either through some form of (annual) fee, or perhaps factoring in their review work to the workload allocation model.
• There needs to be buy in. This is crucial: any new system has to have the backing of the PVC, the Heads of School and the Directors if Research. If it comes just from the centre, or is seen to be nothing more than a bureaucratic burden, then it's doomed. Moreover, or needs to be seen by the applicants as relevant and 'on their side'. Which brings me on to the final point:
• It should be Faculty/School based. This surprised me, and I don't think I agree with it. However, I understand the point being made: it needs to have ownership by the academics. If it's university wide there's a danger that this will be lost. This may be true at bigger universities, but I feel that at Kent the academic base is small enough for this not to be a problem. In addition, I tried to incorporate some of the school systems that already existed before our peer review system was launched. Nevertheless, we need to be alert to this as a potential issue.

And it's that kind of thing which makes involvement in these kinds of events worthwhile. Although I'm at the front pontificating and pretending I have all the answers, I've got as much to learn as anyone.

A Decade of Development

Research Fortnight asked me to write an article to coincide with the 2013 ARMA Conference. I've reproduced it below (thanks to John Whitfield for this), but you can see the original on their website, here.

_________________

Spoiler: Neil Young doesn't feature
in this article. At all. 

Ten years ago, Iraq had just been invaded. Tony Blair was still in office. The iPhone hadn’t been invented. John Peel was still alive.

Ten years ago, I started working as a research funding officer at the University of Kent in Canterbury. University research offices up to then had essentially been branches of finance: they acted as accountants, costing applications and managing awards. Anything else—an occasional visit from a funder, a termly newsletter—was an add-on.

The introduction of a research development service was intended to provide more proactive support to researchers at the university. We could do whatever we thought necessary to help staff improve the quality and quantity of applications. Here was a service that aimed to change the role of university administrators: to have them stand beside academics, take them by the hand and lead them through the funding maze.

In the decade since, research development has become embedded in higher education, transforming the relationship between academics and administrators. It is now unthinkable that a research-intensive university would not have a research development team, says Adam Golberg, a research manager at Nottingham University Business School. “An institution where researchers have no access to specialist information about funders, and no-one sending them possible leads, is not one that is serious about research,” he says.

Each university has defined the job in its own particular way. Some focus on supporting applications, others take a more strategic view; some have a dedicated research development officer, whereas others combine the role with costing and contract duties. David Young, research funding manager at Northumbria University, sees funding development as falling into two categories: direct and indirect. “Direct support,” he says, “is about one-to-one or group-based co-writing support for particular grant applications. Indirect support covers the whole framework or environment surrounding research bidding activity—such as mentoring schemes, training programmes and mock panels.”

Development is still a relatively small part of research support. Jon Hunt, deputy director and head of the Research Development and Collaborations team at the University of Bath, estimates that such roles account for about 10 per cent of the team’s service. However, though the team gets involved in only a fifth of grant proposals, they “have supported nearly 40 per cent of the value of this year’s awards—around £20 million”, he says.

Even so, it’s difficult to attribute a successful application to a development office’s input. This makes practitioners wary of taking the credit. “It is hard to attribute increases in success rates to any one element,” says Linsey Dickson, research funding and liaison manager at Heriot-Watt University in Edinburgh.

For Justine Daniels, head of research development at the University of Sheffield, proactive engagement with funders is vital. They discuss schemes and applications and listen to feedback “that principal investigators can’t always hear”. As a result, Sheffield’s Biotechnology and Biological Sciences Research Council applications have become more in tune with the council’s priorities, success rates have improved, and 20 Sheffield researchers hold fellowships from the European Research Council, up from four in 2010.

Daniels thinks that research development will increasingly become linked with post-award activity. As funders aspire to become sponsors, rather than distant bodies distributing funds, universities need to build on a relationship that begins before an award, and project managers will need to interact more closely with funders. Hunt agrees: his team has both research development managers and a research project management service that works on 14 projects worth around £17m.

Research development is a child of these straitened times, its bullish rise shadowed by the bearish withering of the economy. Budgets are tighter, and schemes have been axed or restricted by demand management. In this climate, research development has not led to a great leap forward, but it has prevented a slow slip backward.

At Kent, I believe the funding team has fostered a more positive research environment, from supporting individuals to developing an internal peer-review system. This has led to some notable wins, but more importantly has kept research—and research funding—at the top of the university’s agenda when the shutters outside have been banging in the cold wind of austerity.