Is There Something Missing from ROS?

'We've got a record of  the number of bodies.
Who cares about the original bodies?'

As many of you will be aware, the Research Councils (RUCK) introduced the 'Research Outcomes System' (or ROS)  to monitor the outputs of their funded projects. The new system was launched on 24 Nov 2011, and took the place of Annual Reports and End of Award Reports, simplifying the wordy demands of its predecessors so that investigators need only note what publications (and other outputs) have resulted from a grant.

On many levels this makes sense, and RCUK aren't shy of crowing about the benefits:

• It saves time. RCUK suggest this could be as much as 80% compared with the previous system.
• It can be used at any time during or after the grant.
• Multiple users can view the reports. 

And so on. All very good. But what struck me the other day is that the new system seems to miss out a crucial element of the End of Award Reports: monitoring and assessment. Whilst RCUK annual monitors compliance with ROS, there doesn't appear to be anyone checking the quality of the outputs or the success of the project measured against it's original aims and objectives.

Or am I missing something?

This seems to be a considerable oversight. When I worked at the AHRC, getting academic reviewers to assess the success of the project was a laborious but crucial element of the grant cycle. If a project was deemed to have failed to produce the intended goods, or had veered too widely from the original path, it could be labelled as 'unsatisfactory', and would have a limited time to come good. If it still didn't, the investigators would be barred from applying again to the Council.

Equally important, if the project had been seen to succeed beyond anyone's wildest dreams, it could be labelled as 'outstanding', and the investigators could bask in the glow of approbation.

Okay, so the old system may have smacked of the schoolroom, but I think it is important that investigators are held to account - for better or worse. The new system just seems to be a bit, well, quantitative.  As long as you give them a title or two, it doesn't really matter beyond that. The spaces are filled, the box is ticked, the numbers are counted.

As I say, I could be wide of the mark on this, and I'd welcome any thoughts from those in Death Star House as to the level of assessment that goes on post-project.

Everything's ROS-y at NERC

NERC has decided to join AHRC, BBSRC, EPSRC and ESRC and join the Research Outcomes System (ROS). This will take over from its Research Outputs Database (ROD), which they've been using for nearly a decade.

The Council make the case that this will:

• Reduce the reporting burden by reducing the number of equivalent systems;
• Simplify submission by moving to a more standardised questionnaire;
• Improve how publications are handled;
• Share information better between systems, reducing data entry and reducing transcription errors; and
• Improve the quality of performance information available to support the case for public investment in the environmental sciences.

What's not to like? Well, as reported here a few months back, the new system isn't without glitches. However, NERC isn't adopting it immediately. Oh no. As in the Life of Brian, this calls for immediate...discussion. NERC will bound into action by:

• Completing the current collection exercise on the existing system;
• Establishing a project to manage the process of adopting ROS for future years collection with Centre participation;
• Engaging with users through the project to ensure that user requirements are identified and met;
• Adapting ROS where necessary to address NERC requirements, including coverage of grants and Centre programmes; and
• To migrate, as necessary, historic data.

I love this, the snail-like progress of bureaucracy in motion. It is a thing of beauty. Now if you want more information as to where we are in the programme of NERC migration to ROS - and who doesn't? - it can be found here.

RCUK Research Outcomes Project Goes Live

RCUK has been developing a system to collect data on outcomes from all their funded research. I highlighted it in January last year, and it's development has not been without controversy. In June last year Research Fortnight reported on disquiet within universities about the additional work that would arise from inputting data on to the system. There was also some concern about how it would fit in with or replace the MRC's existing e-Val system, which had only recently replaced the Output Data Gathering Tool (ODGT).

Anyway, it all seems to have been settled now, and we've just had word through that it is finally going to go live. Details are available here, with some FAQs here (pdf). It will be released to research offices in the week commencing 24 October 2011, to give us time to become familiar with the system, and feedback any queries or concerns to RCUK.

It will then be released to all Research Council grant holders in the week commencing 14 November 2011. The system will collect details of the following:

• Publications
• Other Research Outputs
• Collaboration
• Communication
• Exploitation
• Recognition
• Staff Development
• Further Funding
• Impact

If you have any questions about this, do get in touch with me.

RCUK Announce New End of Award Report System

RCUK has launched it's 'Research Outcomes' system project. This has been rumbling on for some time, partly because the Research Councils recognised that the current system isn't fit for purpose as it doesn't capture outputs that are more than a year or two after the end of an award. The idea, from what I understand, is that award holders will complete a simplified end of award report, and then will be responsible for updating the system as and when new outputs resulting from their funded research become available.

They're currently, they say, consulting with the research community 'to establish requirements for the new system, and the project team is working closely with HEFCE to align, where possible, with similar requirements for the Research Excellence Framework (REF).' Joined up thinking? Surely not!

RCUK will then invite tenders to develop and implement a system, and the project is expected to have a pilot system in operation by summer 2011, with full roll out in late 2011.