This blog is about the archaeology of death and burial, but in the UK, our research on these areas takes place under the shadow of an exhausting and oppressive government audit of our research: the REF.
The REF. Love it, hate it, you just can’t ignore it if you work in higher education in the UK.
Sorry let me start that again.
The REF. Loathe it or hate it, you just can’t ignore it if you work in higher education in the UK.
I have just completed the draft of all the documents as coordinator of the University of Chester’s REF Archaeology submission, so I thought it might be a good time to tell the wider world some things about the REF process and some general issues and opinions that many academics share regarding what the REF is doing to academia and archaeology specifically. I don’t want my comments to be seen as a criticism of my particular University, but comments on a shared experience relevant across the UK from the perspective of an archaeologist.
Background: The RAE
In the UK we used to have the RAE (Research Assessment Exercise), now replaced in 2014 with the REF (Research Excellence Framework). In my short academic career to date, I have contributed as a newly appointed lecturer to the RAE in 2001 and then still as a junior academic to the 2008 RAE.
What are these? Well, for those not in the UK or not in academia, they are government-led exercises to assess the quality of the research being conducted in UK academic institutions and allocate research funding accordingly.
REF 2014 Outputs
The REF is a new exercise in many ways. 60% of the unit of assessment is an appraisal of ‘outputs’ (publications in the form of books, book chapters and journal articles). Established academics are expected to have produced a minimum of 4 within the audit period from 2008 to 2013. In fact, most of us produce far more than this, but we are supposed to pick our ‘best’, whatever that means. How do we pick our best? Well, most UK ‘units of assessment’ will appoint a ‘REF tsar’ (I am probably the only one who calls them that) who will oversee their unit’s REF submission. For archaeology at Chester that is me!
Our outputs are supposedly judged on their merits, but whether venue alone, ability to be ‘internationally significant’ (whatever that means) and different academics’ notions of originality, scholarly rigour and perceptive insight, or simply ‘finding new stuff’. In reality, we look for quality in a wide range of factors, but it remains a very tricky business guessing what the REF panel will think of particular papers and how our perception fits their criteria for what constitutes ‘excellence’. What is also clear that some of our best work is simply not appropriate for the strict criteria of what constitutes ‘international’ research.
I can only speak about this from a personal perspective. For me, I had 20 research papers (book chapters and journal articles) that I believed illustrated my intellectual labour during the audit period 2008 to 2013. I had to pick 4. Which 4? In the audit period, I have produced 11 book chapters, the best of which are easily good enough for REF submission being through rigorous peer-review and published by internationally renown academic publishers. But we are told book chapters aren’t rated as highly as journal articles as a rule. So I have decided to submit four of my journal articles. Of these journal articles, I have 9, all published in peer-reviewed international journals. I have picked 4 that I think show off the range of my work. But frankly I won’t know whether the panel would rate some of those I haven’t selected higher.
Anyway, I overlooked some of the work I am most proud of, and went for a mix: one paper on the early history of Anglo-Saxon archaeology in the mid-nineteenth century, one on the reinterpretation of Mound 1, Sutton Hoo, one about gardens of remembrance in contemporary Swedish churchyards, and one about the National Memorial Arboretum. Sadly this selection meant I decided not to submit papers on Anglo-Saxon cremation urns, papers on death and memory in contemporary animal sanctuaries, a chapter on Viking boat-graves and an excavation report on the same. In other words, for me, REF doesn’t allow me to show off even a fraction of my research efforts and therefore is not appraising most of what I do.
If my job was simply about serving the REF, I could have produced c. 20% of what I have produced since 2008. One might say that I could have put my efforts into producing less, and producing it better. This is true, but it is difficult to know what else I could have done to improve what I have produced and the places I have published it, given that most are premier venues for the subjects in question. Still, everyone has an opinion and a judgement to make on these issues.
REF 2014 Environment
20% is about ‘environment’, the facilities, support, and community within which the research takes place, including aspects of ‘esteem indicators’ (keynote lectures delivered, conferences organised etc), editorial positions and grant income received. This section also includes the postgraduate research students associated with the unit.
For me, this was the easiest section to write because I could tell the REF about a wide range of activities we are involved with, and talk about all six of the archaeologists involved in the REF exercise at Chester. But here there were niggles for me too. Conference organisation is rated, but session organisation within an established annual conference – something I have put a lot of my time into doing and has resulted in numerous networks and dialogues with academics worldwide – doesn’t get recognised. Moreover, there is little way of showing off the detail of the range of micro-activities that take up so much of my time and that are integral to supporting colleagues and students in conducting research.
REF 2014 Impact
While tweaked and re-arranged, the outputs and environment sections are familiar to those who endured the RAE. The new and most challenging element is a further 20% of the assessment which appraises ‘impact’. This has caused most objection and concern. Endless discussions have taken place of what this ‘impact’ is. The ‘impact’ is on society; research that leads directly to a benefit; material, emotional or social. We don’t have to show all our research has ‘impact’, just that, among a group of academics, some of us are doing research that has had some kind of benefit to people today. Moreover, impact is only a fraction at present, 20%, of the overall assessment of any unit of academics. Still, the obsession with identifying, discussing and appraising impact never seems to end. Indeed, some archaeologists have put considerable efforts in their time, some to good effect, in designing entire archaeological projects and activities, digging up English monarchs, lobbying government, in a clear effort to show how their research is ‘relevant’ and has ‘impact’. Good for them!
For Chester, we had to write two impact case studies and we decided to do this by focusing on our fieldwork. With only two longer term academics in position, it fell to Meggen and myself to each construct impact case studies. Meggen chose her Rhynie fieldwork which has produced a unique and internationally important Pictish high-status settlement site. For me, my Project Eliseg fieldwork with colleagues at Bangor will serve, showing how we uncovered new information about the location and context of a distinctive early medieval monument with a long ninth-century Latin inscription. The problem is, both show the promise of impact but will they satisfy the panel as REF impact statements? Have we ‘changed people’s lives’? Are we ‘relevant’? Have we enriched people’s connections to the past and place? We’ll wait and see.
Most academics hate the REF. They despise it for a wide range of interlinked reasons. Some idealists see it as a waste of time and irrelevant to their job. It might be said that it is something for the pen-pushers looking for justification of where tax-payers money have gone. Some see it as one among many tedious administrative tasks that weigh academics down and prevent them from doing their job of researching and teaching.
Others accept the important principle that the government should see where tax-payers money is going and allocate funds to support the ‘excellence’ of the best institutions and scholars. However, there are questions over the criteria and weighting of what gets assessed, as well as the assessment process itself. Other cynics would point out that it is an expensive way of legitimising and extending a hierarchy between research and teaching-focused institutions that is pernicious and retrograde; giving the fat cats of the Russell Group excuses to cull departments that were hitherto seen as success stories but simply don’t ‘perform’ in the REF game and focus on those departments and institutions who can swallow even more of the increasingly diminishing pie of government funding. Others still would point out that the frenzy and pressure to publish can result in substandard research publications. Others again would point out how only a fraction of what academics do in terms of research seems to count as ‘quality’ research in the REF. And again, many would cite the destruction of morale and the divisive effects of pitting academic against academic, dept against dept, in the horrid process of compiling and selecting staff’s outputs and impact statements. REF affects and destroys morale and careers. Whether it improves academia and improves standards and quality of research is difficult to see.
Then there is the way in which the REF is seen by many academics as dictating not only the timing and quality, but also the content of research. We need projects that the public can like, play with, find ‘relevant’ to their 21st-century needs. When my former supervisor told his German counterparts of the system in the UK, one remarked to him in shock how it wasn’t even that bad in the DDR! REF affects intellectual freedom and how this freedom works in daily practice and in publication strategies.
And the REF is seen by many as not very sympathetic. It does take into account some kinds of personal circumstances that interfere with research, but not others. Maternity leave gets recognised, and long-term illness. There are ways it can account for complex circumstances but certain key life-related issues are not easy to account for, including motherhood of young children or paternity (for which the UK has a dismal level of support for).
And of course, what is the reward for doing well at the REF? Is their a financial or prestige award? No, you get to keep your job and perhaps feel smug about those that haven’t done quite so well just as a way of coping with the situation.
I could go on. The point is that the REF is widely seen as a distorted, biased and destructive assessment process that doesn’t suit anyone apart from a minority of academics who are already positioned to do well.
Defending the REF 2014 exercise
But that is only part of the picture. Having said all that, I think the balance of 60/20/20 is about right and the combination of sections allows Chester archaeologists to show off much of what they do. It may not bring us money because we are still a young and small subject area at the University. Moreover, academics should be recognised and accountable for what they write and what they work on. Who isn’t in public life? There are many good things about the timing and structure of the REF and there remains considerable latitude concerning what, how and when we research what we want. And yes it is an exhausting administrative process, but it is far less exhausting that many of the other processes to which we are subject in UK academia and far more purposeful.
For Chester, it has an even greater importance. We don’t anticipate coming top of the league. But it is very important that, for the first time, we put in a submission. It tells the academic world that Chester has arrived on the scene as a young, vibrant and very motivated and new research hub for archaeology. We have outputs, environment and impact to show off. We have exciting field-based research, exciting new ideas and exciting new approaches. Whatever its problems, REF has involved a lot of work, a lot of grief and a lot of support required from my colleagues and Head of Department. But in the end, it is a reality about what we do. We may not like all aspects of the game but it is a game we have to play. My job has been to try and ensure we play it the best we can as a small, new research unit that I believe has a very bright future.