UK’s Research Excellence Framework (REF) & Philosophy

UK’s Research Excellence Framework (REF) & Philosophy

A few days ago, the results from the UK government’s 2014 Research Excellence Framework (REF) exercise were released. I suspect that a lot of non-UK readers are largely unfamiliar with the REF, so first some basics:

The REF is used by the government to assess the quality of research in UK higher education institutions. It replaced the earlier Research Assesment Exercise (RAE). According to the REF website:

  • The four higher education funding bodies will use the assessment outcomes to inform the selective allocation of their grant for research to the institutions which they fund, with effect from 2015-16.
  • The assessment provides accountability for public investment in research and produces evidence of the benefits of this investment.
  • The assessment outcomes provide benchmarking information and establish reputational yardsticks, for use within the higher education (HE) sector and for public information.

There are three elements that are assessed:

1. Outputs: publications. For each person submitted, a department must (normally) submit 4 publications.  Departments can choose to not submit people, and this time around many did this—which is every bit as unpleasant as one might imagine, for everyone.  Departments can also submit anyone who’s on even a 20% contract, and get full credit for their outputs.  Some did a lot of this.
2. Impact: non-academic impact of research done in the last 15 years.
3. Research environment: the “vitality and sustainability” of the particular departments or other units.

All elements (right down to individual publications) are ranked as 1*-4*, with 4* as best. (Thanks to Jenny Saul for this simple explanation.)

On the following table (from pages 34-35 of a PDF available at the Times Higher Education Supplement), there are three rankings: overall average, just outputs, and just impact.

REF chart philosophy 1

There are also other ways of ranking.

Michael Otsuka wrote in recommending a different formula which does not give large departments a “gratuitous advantage simply for being large.” He suggests one “that just takes the percentage of a Department’s overall quality profile ranked 4*, multiplies that by 3, and adds that to the percentage of that Department’s overall quality profile ranked 3*. This reflects the government’s current formula of funding just 3* and 4* research, with 4* research funded at 3 times 3* research. But it treats all quality profiles of Departments as equal, however many people were submitted.”

The ranking Otsuka’s suggestion generates is:

1. University of Oxford
2. University of Birmingham
3. King’s College London
4. University College London
5. University of Cambridge HPS
6. University of St Andrews
7. London School of Economics and Political Science
8. University of Edinburgh
=8. University of Warwick
9. University of Essex
10. University of Sheffield
11. University of Leeds
12. University of Bristol
13. University of Cambridge Phil
14. Birkbeck College

(Raw data is available here.)

Funding decisions by the UK government will be based on the REF, but my understanding is that it is at this point unclear which particular ranking will be used to determine the allocation of funds. Discussion of any aspects of the REF and its use are welcome. (Posted by request.)

There are 8 comments

Your email address will not be published. Required fields are marked *

Please enter an e-mail address