The Liverpool View: Living with Research Assessment

Thomas Harrison is Rathbone Professor of Ancient History and Classical Archaeology in the University of Liverpool’s Department of Archaeology, Classics and Egyptology

“In one of the Letters from London in which Julian Barnes described British life and politics for readers of the New Yorker, he illustrates the impact of Mrs Thatcher’s ideology on our lives through the example of carol singers.

At the time that Thatcher came to power, they would stand outside your house, sing a few carols, and then ring on your bell to see if you wanted more. ‘Halfway through the rule of Thatch’, they began only to sing when they had checked that you ‘were there to listen and pay up’.  After a full decade of Thatcherism, Barnes peered out of his house one evening to see two boys at a distance. ‘”Carols?”, one of them asked, spreading his hands in a business-like gesture.’ It was as if ‘he had just acquired a job lot of tunes off the back of a lorry and could perhaps be persuaded to cut me in’.  Thatcherism was not a reversible phenomenon, Barnes then realised, but ‘an irradiation of the soul’.

Measuring academic research

The first attempt to measure the quality of British academic research – as “outstanding”, “above average”, “average” or “below average” (even those terms seem quaint now) – took place at the height of Thatcher’s dominance, in 1986.  And just as Thatcherism represented, in Barnes’ image, not just a pendulum swing but a rehanging of the whole clock at a different angle, so research assessment in all its guises – the Research Selectivity Exercise, the RAE (Research Assessment Exercise), now REF (Research Excellence Framework) – have transformed academic life in ways of which we are not always even conscious.  No longer just a periodic ‘exercise’, but a ‘framework’, it structures our daily lives.

I’ve grown up with research assessment, as I grew up with Thatcher.  First, I prospered by it.  My first permanent job came at a time when I was (in terms of publications) like a full-fed calf ready for market.  Since then I’ve operated as a low-level enforcer of this research economy, trying to ensure that our plans for research ‘outputs’ (i.e. publications), ‘research environment’, and the verifiable ‘impact’ of our work outside academia are fulfilled in spite of all the obstacles of work and life, and drafting and redrafting innumerable words of fact-crammed strategic prose.

As a REF coordinator married to another REF coordinator, moreover, in the run-up to the recent REF deadline  there was no escape even at home.  A rare evening of not working saw us collapsed in front of a documentary in which academic vets tracked the movement of domestic cats around a Home Counties village. One by one, villagers pronounced that their understanding of their seemingly sleepy Mog had been transformed on discovering that he or she had been marauding far and wide round the village.  Instantly we turned to each other: ‘if only we had such clear evidence of impact’ the unspoken thought.

Until now, I’ve always been – at least in part – an apologist for research assessment, not least for its effect in ring-fencing some time for research.   But, more and more, I wonder.

First, there is the clear evidence of the negative impact on gender equality (http://www.timeshighereducation.co.uk/features/sex-and-the-ref/2012076.article).  Or the immense damage done to individuals’ morale and to the cohesiveness of a department when a given colleague’s contribution is not deemed worthy of inclusion.  (If I still have the respect of some of my colleagues, have I not done a good enough job?)  Then there is the false wedge that REF has driven between books for a wide audience and weighty ‘REF-able’ research.  (I doubt I am alone in sometimes hearing an inner voice, telling me to slap on a few more footnotes for good measure…) 

And then, what end does this industry serve?   From one cycle to another, we’ve all become increasingly professional players of the game. (A colleague remembers the proud claim made in a 1996 submission, at another University, that ‘the department has its own photocopier’ – unthinkable now.)  We are all looking for marginal gains, in competition against colleagues and friends in our own disciplines, for a diminished pot of money.  And, all the while, our main focus has shifted to the recruitment of fee-paying students.

Perverse incentives

Above all, perhaps, for me, it is the perverse incentives of this world of research assessment, its artificiality, and the sense of dislocation between the daily business of research and its marketization that disturbs.  Seeing the prize-winning products of the best US universities ruled out of contention for UK jobs, for lack of sufficient ‘REF-able’ outputs.

Finding oneself recommending that a colleague or graduate should delay the publication of her monograph till after a REF ‘census date’.  And then the occasional realization of just how deeply embedded the ‘values’ of research assessment are inside myself, of the effort that is needed to remember the real value of research.

My wife thought that a year of reading nothing but poetry might be a cure – not that that was ever feasible.  I worry that the irradiation is irreversible.”