The Research Excellence Framework (REF), undertaken by the UK’s four HE funding bodies, provides a review and assessment of research in higher education. The results of the latest assessment (REF2021) were published on 12 May 2022.
Here, Professors Mark Jackson and Margot Finn — respectively chair and deputy chair of the History sub-panel for REF2021 — offer an overview of this latest review, its headline findings for History, and their reflections on disciplinary developments since REF2014.
Since 1986, UK-based university historians have participated in periodic assessments of research excellence. Previously denominated the Research Assessment Exercise (RAE) and now known as the Research Excellence Framework (REF), this process typically occurs at 6-7 year intervals. The two most recent iterations have been REF2014 and REF2021.
The results of REF2021 were announced on 12 May 2022 and can be accessed here.
Results for the History sub-panel for REF2021 are available here.
About REF and REF2021
UK REF exercises have 3 main functions:
- They are used by the government research funding bodies of England, Northern Ireland, Scotland and Wales to allocate Quality Related (QR) funding to universities. Institutional QR allocations reflect the number of staff entered in the REF by the given university, the quality thresholds attained by its combined Unit of Assessment (UoA) submissions (History, Classics and Archaeology, for example, are each UoAs), and the subject mix of the university’s UoAs (historically, funding for STEM UoAs has been higher per staff member than Humanities funding, for example). Annual QR allocations based on performance in the most recent REF continue for the 6-7 years between each assessment exercise and the next. QR funding is intended to provide institutions with stable income to be used strategically by Higher Education Institutions (HEIs) in order to enable university staff and postgraduate research students (PGRs) to engage in day-to-day and long-term research activities, regardless of whether they are in receipt of external funding from grants.
- REF results are reviewed and discussed within universities and within the UK research councils, and inform research planning, staffing and other strategic activities undertaken by these bodies as well as by governments, charities and other research funders.
- REF exercises provide data that are included in university league tables. As these data are multiple and amenable to varied forms of analysis, they lend themselves to multiple (and often conflicting) claims as to which universities and which discipline-based UoAs are ‘best’ either overall or in terms of Outputs (that is, publications), Impact or Environment.
In REF2021, the overall Quality Profile for each submitted UoA comprised Outputs (60%), Impact case studies (25%) and Environment (15%). Research quality was assessed according to the following scale:
- 4* denotes research that is world-leading, impact that is outstanding, and environments that are conducive to producing research of world-leading quality and enabling outstanding impact;
- 3* denotes research that is internationally excellent, impact that is very considerable, and environments that are conducive to producing internationally excellent research and enabling very considerable impact;
- 2* denotes research that is recognised internationally, impact that is considerable, and environments that are conducive to producing internationally recognised research and enabling considerable impact;
- 1* denotes nationally recognised research, recognised but modest impact, and environments that are conducive to producing nationally recognised research and enabling recognised but modest impact;
- U denotes research, impact and environments that fall below the standard of nationally recognised
These definitions of excellence were shared by all of the disciplinary Sub-panels and the 4 Main Panels of REF2021. History was Sub-panel 28, forming part of Main Panel D (Arts and Humanities). The table below shows the overall weighted profile and the average profiles – weighted by FTE – for Outputs, Impact and Environment for Sub-panel 28.
| % 4* | % 3* | % 2* | % 1* | % Unclassified | |
| Overall quality | 43 | 37 | 18 | 2 | 0 |
| Outputs | 40.5 | 37.8 | 20.4 | 1.2 | 0.1 |
| Impact | 45.3 | 34.5 | 16.4 | 3.7 | 0.1 |
| Environment | 48.0 | 40.9 | 10.2 | 0.9 | 0 |
REF quality assessments are made public at the level of individual universities and their submitted UoAs, not at the level of individual researchers. Neither individual staff nor their employing universities have access to the Output scores attributed to named, individual staff members; instead, they have access to the overall profile of each UoA, generated by its submitted staff collectively. Likewise, published scores of Impact Case Studies are not attributed to individual named Impact Case Studies in published REF results; instead, the overall profile of all case studies submitted by a UoA is reported. Indeed, all of the underlying data contributing to the final published profiles is destroyed after the assessment period; no data relating to individual scores – whether Outputs or Impact Case Studies – is retained.
The History sub-panel judged the overall quality of submissions in 2021 to be higher in many respects than in 2014. History is a strong and vibrant discipline across the university sector, with world-leading research and outstanding impact present in nearly every submission.
It is essential to note that changes to the submission rules preclude direct comparisons between many aspects of REF2021 and REF2014. For example, REF2014 was ‘selective’, allowing UoAs considerable flexibility as to which eligible staff they submitted relative to REF2021, and the weightings and/or content of the component parts of each exercise differ in key respects. Nevertheless, the History sub-panel judged the overall quality of submissions in 2021 to be higher in many respects than in 2014. History is a strong and vibrant discipline across the university sector, with world-leading research and outstanding impact present in nearly every submission.
This remainder of the post reflects on some of the main processes and outcomes of the History sub-panel in its assessment of Outputs, Impact, and research Environments. Fuller details are provided in the Sub-panel and Main Panel D reports published this month. Over the summer, REF will place further information in the public domain on its website. This will include the text of all submitted Impact Case Studies and Environment statements, affording the discipline a rich body of information about historical practice in the UK.
Assessment in the Covid context
Undertaken in a pandemic, assessment for REF2021 was unlike any other. Across the sector, COVID disrupted the publication of outputs and delayed or cancelled impact activities. In addition, it placed considerable burdens on institutions and units preparing submissions, as resources and time were redirected to support staff and students, to continue delivering high-quality undergraduate and postgraduate teaching, and to assist in responding to the pandemic. Successful submission of Outputs, Impact Case Studies and Environment statements in the face of these challenges is a testament to the resilience and quality of academic and professional service colleagues across the sector.
The pandemic also reshaped the assessment process itself, shifting the timetable of assessment, altering the format of meetings, and complicating the development and release of profiles, sub-panel reports, and HEI feedback. Responding to these challenges, the sub-panel took COVID statements into account in its assessment of all aspects of the submissions. This consideration operated in addition to the Intention Plan we developed to mitigate against unconscious bias and ensure fair assessment of all submissions. The successful completion of the assessment phase of REF2021 speaks to the diligence, professionalism and integrity with which members of sub-panels and main panels conducted – and were committed to – the process.
Summary of History UoA submissions
Sub-panel 28 received submissions from 81 UoAs based in universities in England, Northern Ireland, Scotland and Wales.
In all, the sub-panel assessed the research of 2,360.21 FTE staff. With some exceptions for special circumstances, UoAs were required to submit no fewer than one and no more than five Outputs per submitted staff member (combining to 2.5 Outputs per submitted staff FTE across the UoA’s submission as a whole). A total of 5,766 outputs were submitted by the 81 History UoAs, of which the sub-panel assessed 4,418. (The difference of 1,348 primarily reflects UoAs’ ability to request double-weighting of very substantial outputs, naming a reserve item for assessment in case the Sub-panel did not accept this request, thus reducing the total number of Outputs ultimately read for assessment.)
Each UoA submitted a minimum of two Impact Case Studies; larger UoAs submitted more case studies, based pro-rata on the number of submitted staff members. The History sub-panel assessed 248 Impact Case Studies. Each of the 81 UoAs submitted an Environment statement for assessment; the length allowed for Environment statements depended on the number of staff submitted by the UoA, with shorter statement for smaller units.
Outputs
Three criteria – originality, rigour and significance – were used in scoring REF2021 Outputs. For this aspect of the assessment, Sub-panel 28’s 21 members were assisted by seven Output assessors, appointed to extend the sub-panel’s expertise. Outputs were allocated according to the expert knowledge of sub-panel members and Output assessors, taking into account conflicts of interest.
It is important to stress that all forms of Output – monographs, journal articles, edited volumes, book chapters, scholarly editions, websites, working papers and publications in other media – were capable of being judged as world-leading or internationally excellent. In addition, all forms and places of publication were treated equally: the sub-panel did not rank journals or publishers in any way.
Double-weighting requests were made for a wide range of Outputs, but especially for authored books, reflecting the importance of monographs and other Outputs of extended scale and scope to History as a discipline. The great majority (99.1%) of double-weighting requests were accepted. A small number of submitting units did not request double-weighting for some of their submitted Outputs, despite the submission of Outputs that would have passed the threshold. The decision not to request double-weighting may have been an opportunity missed, given that reserve items offered a safety net if a double-weighting request was rejected.
A total of 258 Outputs were cross-referred to, or jointly assessed with, other sub-panels, either where it was requested by the submitting unit or where it was judged that the Outputs concerned were more appropriately assessed outside Sub-panel 28, especially by other sub-panels within Main Panel D, but also in some cases by those in Main Panels B and C. 112 outputs were cross-referred into Sub-panel 28 from sub-panels in all four main panels. The flow of cross-referrals and joint assessments into and out of Sub-panel 28 testifies to the ways in which historical research informs, and is informed by, scholarship across a broad disciplinary range, including STEM subject areas.
In many cases, cross-referral testified to the inherently interdisciplinary nature of historical scholarship. The sub-panel assessed a substantial amount of work that incorporated methods and insights from other disciplines, from medicine through social sciences to other arts and humanities disciplines. The range of quality shown in interdisciplinary research was the same as in Outputs that were situated more clearly within the discipline.
Research of outstanding quality with respect to originality, rigour and significance was evident in both established and emergent fields of historical research. Outputs that explored topics using comparative, international, transnational or global perspectives often scored highly.
Outputs demonstrated areas of world-leading quality across a wide variety of subjects, periods, and methodologies. Research of outstanding quality with respect to originality, rigour and significance was evident in both established and emergent fields of historical research. Outputs that explored topics using comparative, international, transnational or global perspectives often scored highly. The sub-panel noted that high-quality research in these and other areas of scholarship often requires a good command of languages other than English, which was not always apparent. In some areas, including medieval history and ethno-historical work in the Indian sub-continent, Latin America and Africa, multi-lingual research was evident, but in some others there was a lack of engagement with archival, primary, and secondary sources in languages other than English. This remains a challenge to be addressed over the coming years.
Impact
This was the second REF exercise to evaluate the impact of UK research.
For the purposes of assessment, the quality of the impact of historical research was judged in terms of reach and significance. In REF2021, reach was defined as the extent and/or diversity of the beneficiaries of the impact; significance as the degree to which the impact activities enabled, enriched, influenced, informed or changed the performance, policies, practices, products, services, understanding, awareness, or well-being of the beneficiaries.
Assessment of the 248 Impact Case Studies submitted to Sub-panel 28 was assisted by the inclusion of two user members in the sub-panel and by the appointment of seven impact assessors with a wide variety of experiences and expertise in education, museum and curatorial practice, heritage, the media, public engagement and policy. The participation of impact assessors, who were involved in all aspects of calibration, assessment and moderation, significantly enhanced the sub-panel’s ability to assess the impact of university-based research.
Impact was evident in case studies that engaged with a wide variety of beneficiaries. The largest single category of Impact in History cases studies was heritage and landscape heritage, including work with museums and archives (in total, accounting for over a fifth of submitted case studies). Public commemoration was also a salient category of submitted case studies, notably (but not only) commemorations of the First World War. There were many world-leading submissions focused on public understanding of the past, in some instances including broadcasting media. Impact on well-being, health and social welfare were prominent among submissions, and included attention to social injustice and inequalities, gender and sexuality, and professional and public understandings of health and illness.
Many Impact Case Studies demonstrated contributions to policy in local, national and international contexts through work with government and non-governmental organisations in and beyond the UK. Similarly, a significant subset of case studies focused on political discourse, including debates on terrorism, extremism and Brexit, as well as the history of women’s suffrage; other world-leading case studies focused on justice (for example, civil and human rights) with international dimensions. Outstanding case studies that illuminated the histories of slavery and post-slavery also enhanced submissions to the sub-panel. Many case studies included contributions to education and learning, particularly the development or enrichment of school curriculums.
Outstanding Impact Case Studies were submitted by UoAs of all types and sizes, including smaller units that provided compelling evidence of outstanding impact, often reflecting unit or institutional commitment to supporting impact as well as long-standing engagement with local, regional, and national communities.
The sub-panel was impressed by the strengths and diversity of the impact of historical research. Outstanding Impact Case Studies were submitted by UoAs of all types and sizes, including smaller units that provided compelling evidence of outstanding impact, often reflecting unit or institutional commitment to supporting impact as well as long-standing engagement with local, regional, and national communities. The sub-panel was concerned, however, that the number of Impact Case Studies required for smaller submissions placed particular burdens on very small units that sometimes possess fewer institutional resources to enable impact. Many disciplines in Main Panel D, including History, have a significant minority of submitting UoAs with fewer than 20 staff FTEs, and this burden is especially acute in disciplines with this configuration.
The Impact Case Studies submitted to the History sub-panel demonstrated the variety of available pathways to successful impact. They attest that impact can flow directly or indirectly from historical research. It can stem from a single piece of research or from a wider body of historical work. Impact can be planned as part of a large, challenge-led research project, or emerge in serendipitous fashion during the course of the research. And it can be achieved by researchers themselves or by external organisations and groups. While dissemination of research findings alone is not evidence of impact, public engagement was often a significant component of successful case studies, serving as a step on the pathway to impact.
Many of the Impact Case Studies were interdisciplinary, highlighting the success with which historians work across disciplinary boundaries, collaborating in particular with scholars across the humanities, social sciences and health research to generate impact with global reach and significance. Impact activities were also underpinned by research in a wide variety of historical fields, ranging for example from ancient to contemporary time periods, from local to global, from urban to rural histories, from transnational studies to micro-histories.
Strong case studies clearly demonstrated the link between the underpinning research and the impact and provided a range of sources to corroborate their claims.
Strong case studies clearly demonstrated the link between the underpinning research and the impact and provided a range of sources to corroborate their claims, including, for example, testimonials, audience surveys, feedback from focus groups, visitor numbers, viewing figures, professional testimony, reports, and media coverage. In addition, strong case studies often demonstrated close and sustained collaboration with partners, involving the exchange of expertise and experience and the co-production of impact activities. Collaboration of this nature has been enabled by unit-level and institutional environments that have prioritised the impact of research, as well as the research itself.
Environment
UK university-based historians work in very different environments. History UoAs submitted to REF 2021 varied considerably in size, ranging from 3.5 FTE to 172.8 FTE. They also varied in composition. While some units operate as discrete departments, others, especially smaller units, operate within wider interdisciplinary units or represent multiple groupings of historians from across the institution. Differences in size and structure contribute to the richness of the discipline, with excellence manifest in widely varying strategies, structures, and policies.
The unit-level Environment statements assessed by the sub-panel indicate that the majority of historians submitted to REF2021 work in environments that are conducive to producing world-leading or internationally-excellent research and enabling outstanding or very considerable impact. The sub-panel assessed Environment statements in terms of the vitality and sustainability of submitting UoAs. Assessors based their evaluations solely upon material contained within submitted Environment statements and did not consider developments subsequent to the census period.
Part 1 of the Environment statement focused on UoAs’ context and structure and their research and impact strategies and was worth 25% of their Environment score. Submitted History UoAs demonstrated a wide range of research and impact strategies. Strong submissions provided indications of how their strategies would be implemented, evidence of clear processes for making their research available through open access and ensuring research integrity, and, where appropriate, evidence of how interdisciplinary research was integrated into the unit’s wider research and impact strategy.
Part 2 of the Environment statement on staffing, PGRs, and Equality, Diversity, and Inclusion (EDI) counted for 30% of the overall environment score, reflecting the importance of support for staff and students within humanities disciplines. Doctoral training and staffing strategies, including study leave, internal funding support and mentoring for staff across the full career cycle, were often world-leading.
The sub-panel was impressed by careful attention to the development of researchers at early stages of their careers, whether doctoral students or postdoctoral researchers. Submitting units were required to provide evidence of their policies, aspirations and achievements in relation to promoting and supporting EDI. This proved to be challenging for some units. Stronger statements on EDI provided clear evidence that support for EDI was embedded across all elements of the research environment and addressed EDI in relation to a broad set of characteristics (rather than, for example, gender alone), with data supported by thoughtful analysis, reflection, and plans for the future. Overall, the submissions demonstrated encouraging evidence on EDI, but also underlined the fact that there remains substantial room for improvement.
Part 3 of the Environment statements focused on income, infrastructure and facilities and constituted 20% of the Environment score. The excellence of research environments within History was evident in data on research income; during the REF cycle, there was an aggregate spend of just under £248 million. Of course, external grant income varies substantially between units and different funding strategies can work for different units. Success in accessing external funds was often facilitated by the availability of internal pump-priming funds and support for grant applications within the institution. Stronger submissions demonstrated how external funding, infrastructure and facilities enhanced staff and PGR research and demonstrated institutional investment in (for example) access to library and archival resources, cultural resources and opportunities for public engagement.
Part 4 of Environment statements, weighted at 25%, focused on collaboration and contributions to the research base, economy and society. Units of all sizes and types demonstrated impressive contributions to the discipline, as well as outstanding collaborations with non-academic communities. Stronger units engaged convincingly with communities beyond the institution, showed a balanced commitment to the discipline through, for example, editorial work and contributions to learned societies, and demonstrated clear commitments to national and international collaboration with a wide range of beneficiaries.
Conclusions
It has been a pleasure and a privilege chairing Sub-panel 28, working with colleagues across the sector, as well as with non-academic partners, to ensure that the remarkable strengths and impact of History are fully recognised. Further reflections on the submissions to REF2021 are provided in the Main Panel D and Sub-panel reports published this month.
But a number of headlines stand out from the assessment process and its outcomes.
Submissions to Sub-panel 28 indicate that world-leading research, outstanding impact, and sustainable research environments are evident across the discipline. Submitting units have shown considerable resilience and creativity in the face of COVID and challenging economic circumstances, enriching and sustaining a vibrant, world-leading discipline across the UK university sector in the process. The extraordinary range of submitting units and the diversity of approaches to historical research and impact constitute outstanding strengths of the field.
The results of REF2021 demonstrate that, although challenges clearly remain, there is much to admire and much to be proud of in our discipline.
About the authors
Professor Mark Jackson is Professor of the History of Medicine and Research Theme Leader for Medical Humanities at the University of Exeter. Mark is Chair of Sub-Panel 28 (History) for REF2021.
Professor Margot Finn FBA is Professor of Modern History at University College London and a former President of the Royal Historical Society. Margot is Deputy Chair of Sub-Panel 28 (History) for REF2021.