Goldilocks Grants

The first, second and third reviewers disagree about the merits
of Goldilock's research methodology

Earlier this year Michael Lauer, the National Institutes of Health’s (NIH) Deputy Director for Extramural Research, wrote an interesting blog post examining the productivity of its funding. He looked for a correlation between the amount of funding a project had received and the number of citations it got. This he described as “citations per dollar”.

Such a stark metric had many rolling their eyes, and the comments that followed questioned the underlying supposition. “Numbers of citations rarely correlate with greatest discoveries”, wrote one commentator. “Citation numbers are strongly biased towards fast-moving areas of inquiry”, said another. A third noted that “the number of citations a paper receives is an extremely error-prone measure of scientific merit.”

Nevertheless, I think it’s worth exploring Lauer’s work further for two reasons. First, because it is entirely appropriate for funders to try to assess the most effective use of their limited resources. Second, because the conclusion he hints at runs counter to the current drive by the majority of funders to back larger and longer projects.

'Citations Are Peer Review by the World'

Prof John Mingers

The research excellence framework - or REF - has become such an established part of the higher education landscape that it’s hard to imagine a world without it. Since 1985 it has provided the navigation points in a turbulent and changing landscape, marking progress and, like a modern day Lachesis, measuring the threads of individual research in readiness for Atropos’ shears.

As we gear up for REF2021 and wait to hear from HEFCE about what form it will take, it seems an appropriate time to take stock and question whether the game is worth the candle, and if the measurement of research should be undertaken differently.

Michelangelo: 'Not Ticked All the Metrics'

Michelangelo: 'a glorious failure'

Whilst there has been some disquiet amongst arts organisations about Arts Council England's plans to introduce a national quantitative system for evaluating the quality of the art it funds, they needn't worry. Such systems have long been used by commissioners of artistic works. A transcript of a discussion between Pope Julius II and Michelangelo has recently come to light that confirms that such 'quality metrics' were already in place when the artist was working on the Sistine Chapel. We reprint it here to put our readers' minds at rest.

Fundermental Top 10 of 2014

Last year I compiled a list of the top ten posts on Fundermentals, based on page views. As we inch towards Christmas, and before the deluge of the REF, I reckon it's time to do the same for 2014.

The Danger of Management by Metrics

The Metric-Measured Man

There was depressing news from Kings earlier this month. The University plans to cut 120 academic posts in the Schools of Medicine and Biomedicine (from a total of 777)  in order to fund new building. Chillingly, the redundancies will be decided by two metrics: research income and teaching hours.

Using metrics is somewhat fraught at the best of times, but is positively toxic when jobs depend on it. The idea of having stated targets is something that all universities have toyed with and many have implemented. There are clear examples where these targets have had a real effect, focussing minds and ratcheting up activity. But at what cost? Anecdotally the morale in these metric-driven universities makes Airstrip One look positively Arcadian.