Article - Issue 24, September 2005
UK Research Excellence: Esteem is not Enough
Dr Ian Nussey OBE FREng
Dr Ian Nussey OBE FREng © Hedley
The Research Assessment Exercise (RAE 2008) will judge the quality of university research in the UK. The results of the assessment will have a major impact on the distribution of funds. Ian Nussey argues that there is still time to improve RAE 2008 so that it recognises outcomes from previous submissions and rewards research that has influenced the quality of life and material well-being.
Few companies in the UK now sponsor or conduct pre-competitive research, a move that reflects investor attention to the bottom-line and boards preferring to allocate pro bono publico budgets to other corporate social priorities. Legitimate concern that investment in near-market R&D is too low has resulted in several national initiatives, including Faraday Partnerships, the DTI’s Technology Strategy Programme, and the Chancellor of the Exchequer’s not very generous R&D tax credits. Though industry self-help is important, we should also ask if there is more that academe can do.
It seems axiomatic that academic research into applied science, engineering and the business sector should contribute to national competitiveness and social well-being. There must then be an unease respecting the findings of the most recent Community Innovation Survey (www.dti.gov.uk/iese/cis.htm). This admittedly slightly dated survey, which sets out to monitor Europe’s progress in the area of innovation, found that firms in the UK cited universities as highly important sources, less than any other EU15 country.
It follows that universities need related incentives. Yet the recently proposed third round of the Higher Education Innovation Fund (HEIF3), responding to the Lambert review of business-university collaboration by allocating £238 million to higher education institutions over two years, seems unlikely to influence upstream research policy. A further point is that too many bids for EPSRC funding include well-intended promises of industrial collaboration and technology transfer that are inadequately honoured.
Supposing that academe has a duty of care to improve matters, there is a way to do so without making a radical shift from research to near-term development. Though my experience is that easy and obvious remedies to difficult problems often fail, it does seem sensible to make a modest adjustment to the latest Research Assessment Exercise (RAE 2008), which is being managed by the Higher Education Funding Council for England (HEFCE) and its sisters. (Anyone considering commenting on this exercise should note that the nicely put together and self-consistent draft RAE 2008 proposal, www.rae.ac.uk, is out for consultation until 19 September 2005).
Universities will have no difficulty making best advantage of the relatively few, perfectly sensible changes from RAE 2001, such as the introduction of sub-group affinity teams. Arguably though, the exercise is still an academic zero sum game, with little chance of adding to the total research pot and minimally interested in making a difference in the world outside. Significantly, just 23 of more than 900 reviewers on the various subject panels come from industry, with around a third of these assessing engineering research.
The 15 subject panels of RAE 2008 have provisionally decided on the weighting that their sub-groups will place on each of the three elements of the review: Publications, Research Environment and Esteem. HEFCE has set minima of 50% for Publications and 5% each for Research Environment and Esteem, leaving individual subject panels to allocate the remaining 40% as they choose.
There are some interesting variations. The subject panel for ‘Economics and Econometrics’, for example, allocates a weighting of just 5% to Esteem. Business and Management Studies (notably without a single industry representative), Applied Mathematics and Biological Sciences allocate 10%, Middle Eastern and African Studies 15%, Physics 20%. Engineering, bravely and commendably, allocates a weighting of 30% to Esteem. At the same time, the engineering panel has uniquely elected to allocate the minimum weighting of 50% to Publications and to give half the Esteem weight to research-related income.
What particularly worries me is the limited, even arbitrary choice of the criteria for assessing Esteem. Fortunately, the engineering panel has taken on board conclusions from two recent reports from the Academy: ‘Wealth of A Nation’ (joint with EPSRC) and ‘Assessment of Quality in Engineering Disciplines’.
Following the latter’s recommendation that the RAE 2008 should ask for demonstrable and quantifiable evidence of the impact of engineering research, all six engineering sub-groups are explicitly asking about the exploitation of work submitted. What a pity other assessment groups for applied science and business have not followed suit.
It must be a matter of public concern that groupings such as computer science and informatics, chemistry, business & management and food science are virtually silent on this point. Are they aware of that Community Innovation Survey? Could it be that they consider beneficial outcomes for business or society to be irrelevant? Or is it that they have too little to report?
Delighted though we should be at the boldness of the engineering panel in giving Esteem so significant a weighting, and its innovative request for evidence of actual outcomes from the current research assessment cycle, that is not enough. It usually takes longer than a single assessment cycle to turn basic or strategic research into beneficial outcomes.
Just as sensible investors calibrate the quality of the financial statements and the chairman’s nice words in the latest annual report by looking back at what was said in previous years, shouldn’t the Research Assessment Exercise do the same? In other words, it should ask for summaries of how the publications that currently active researchers entered for previous assessment exercises have moved the world forward.
So I propose a change, that the Research Assessment Exercise should ask for evidence that previously reported work has influenced quality of life and material well-being. Subjects where this should apply beyond engineering include applied science, engineering, economics, medicine and business. And next time, how about Impact subsuming Esteem?
Revisiting the outcomes of past assessment exercises to assess the impact of previous research would do more than simply add value to the Esteem indicator and influence the ratings in RAE 2008. It could also inform the strategic thinking and deployment of resources by university departments. It could also influence the evaluation of publications, improve related Government decision-making and stimulate tangible industry participation. Companies may well look more kindly on, and invest in, groups that can prove that their earlier research has delivered tangible benefits. Assessing past impact might also help to differentiate between candidates for promotion and give the public an idea of the worth of an important aspect of academic research. While preparing and evaluating such input might be difficult, it would be worth trying.
Biography – Dr Ian Nussey OBE FREng
Ian Nussey was a Lucas graduate apprentice and research scholar. With IBM UK since 1963, he founded and led Warwick Development Group and the company's pre-university student programme. Formerly Chairman of the IBM UK Technical Consultancy Group and a Vice-President of the IBM Academy of Technology, he currently works part-time on University Relations. He is an Honorary Professor at Cardiff and a member of EPSRC's College of Peers.