What do History students think of Gen AI? Over the past academic year, Neville Morley has been working with History and Ancient History students at the University of Exeter to better understand their experience of, and attitude to, GenAI for their studies
In this post, Neville reports on the findings of surveys conducted in 2023 and 2024, charting how students have understood the opportunities, and limitations, of artificial intelligence, and how these may be applied to their reading, writing and research.
Based on these surveys, it appears few History students are using GenAI for their coursework, though there is a widespread perception of its usefulness in generating summaries of individual publications and overviews of topics.
Despite the opportunities GenAI can offer, students still need established historical skills to understand the limitations of this new technology.
Amidst the hype, confusion and occasional panic about the advent of generative ‘AI’ and its potential impact – on the economy, society and culture in general, and on higher education in particular – it’s vital to consider the student perspective: how (and how far) students are actually using such tools, and how they envisage the place of GenAI in their education.
There is little point, for example, in throwing out decades of research in effective assessment practices to revert to in-person exam conditions as ‘AI proofing’ if only a small minority of students are including Generative AI outputs in their coursework.
Conversely, the ways students actually employ GenAI tools like ChatGPT in their studies might negatively affect their skills and their understanding, which historians may find hard to address. Further, we might find variation among disciplines, reflecting different expectations within the university. A recent Policy Note from the Higher Education Policy Institute (HEPI) on students’ views on GenAI considers demographic but not subject differences amongst the students surveyed.
Students who participated in the assessment pilot did offer critical comments on GenAI output when comparing it with their expectations of a ‘proper’ historical discussion.
Over this past academic year, I have run a small project at Exeter on ‘LLMs and Historical Skills Assessment’, funded by Exeter’s Education Incubator scheme. In autumn 2023, undergraduate students in History and Ancient History were surveyed about their knowledge and use of GenAI and their attitudes and anxieties around it. Exactly 100 students responded.
The results were discussed in a series of focus groups. This informed a pilot project exploring how GenAI could be incorporated into the assessment process. We followed up with further questionnaires and focus groups, but found a much lower level of participation.
Knowledge and understanding
In the original survey in November 2023, nearly half of the respondents claimed some experience with GenAI. The focus groups suggested that this amounted to playing around with the tools when they first appeared. Of the students who undertook an assessment incorporating the use of GenAI and reported back on the experience, half had no prior experience while the others had either dabbled or used it only for fun.
Levels of knowledge vary considerably: 40% of survey respondents said they did not understand how GenAI works, and over 70% had no clear idea how or where the AI obtained its data. Just under a third felt that they knew how to get the results they wanted from GenAI. 48% felt that they could distinguish AI-generated text from text written by a human, but focus group discussions showed that this claim was based on questionable assumptions.
Discussion of ethical issues focused entirely on whether GenAI use was ‘cheating.’
Students who participated in the assessment pilot did offer critical comments on GenAI output when comparing it with their expectations of a ‘proper’ historical discussion; overall, they focused on the historical content of the output rather than on its style or the processes that generated it. Those who commented afterwards agreed or agreed strongly that their knowledge and understanding had improved during the exercise; the majority said they felt more sceptical about its current capabilities, but expressed a belief that future versions would solve these problems.
The vast majority of students had learned about GenAI from friends or social media, and felt encouraged to explore it, mostly for entertainment. Several of them, in different focus groups, claimed that their peers in other academic disciplines (predominantly social sciences) were using the tools much more regularly, including for academic coursework. These students were generally sceptical of such behaviour, attributing it to the limited or superficial nature of assessment in other departments.
Discussion of ethical issues focused entirely on whether GenAI use was ‘cheating.’ Exactly half of survey respondents felt that any use of AI in assessment was wrong. There seemed to be no awareness of issues with copyright, environmental impact or biased output, even among students who were interested and engaged with these matters.
Gen-AI and coursework
Around 10% of survey respondents in November 2023 suggested that they would use GenAI to help write their coursework, either to save time or improve quality; in a survey in May 2024, though with a much smaller sample, no students expressed such an intention.
Focus group discussions suggested that there is widespread scepticism about the quality of the output (especially the lack of analysis and argument) and the style (“it doesn’t write like a student”), as well as fear of detection: these students appear to believe strongly in the power of AI detection tools and the capacity of markers to detect malpractice. This does suggest that, as long as assessment criteria ensure an intellectual challenge for our students, AI-generated essays will be quite rare in historical studies. They will usually be submitted by students who resort to GenAI in desperation.
There is much greater acceptance for using AI tools as writing support and as a source of feedback on their work (e.g., by using Grammarly, or by submitting draft essays to ChatGPT). More than 25% of students said that they might use GenAI to generate an essay plan. These students believe that this does not constitute the sort of use that needs to be acknowledged or could be considered problematic. University regulations state otherwise.
LLMs as research tool
There was much more interest expressed in using GenAI to undertake research into historical topics and to provide summaries of scholarship (c. 40% of respondents in November 2023). This attitude persisted among students who completed a critical evaluation of GenAI output.
Most students expressed scepticism about the quality of AI-generated text, but they concentrated on the fact that it was descriptive rather than analytical. Students do not seem to recognise the possibility that some information might be false, misleading or simply unhelpful. What might seem to academics as an obvious risk, that any mechanical ‘summary’ of a topic will entail qualitative weaknesses, and so it does not provide a promising starting-point for research, is not recognised among these students.
It seems that many students regard reading as a task to extract content and not as a valuable analytic skill or worthy of effort.
Particularly worrying is the widespread perception that GenAI can produce useful summaries of articles and chapters, allowing students to avoid reading them in full. Indeed, this was presented in several focus groups as something that could level the playing field for students with learning difficulties like dyslexia, or those with limited time, to allow them to engage with a wider range of material than if they had to read this material without the help of technology. It seems that many students regard reading as a task to extract content and not as a valuable analytic skill or worthy of effort.
Summary
Despite the fears of many in Humanities disciplines that the entire assessment process might be undermined by GenAI, according to this limited sample, few History students use this technology for their coursework. Indeed, they are aware of various deficiencies. However, there is a widespread perception that GenAI can produce useful summaries of individual publications and overviews of topics.
The key issue for History teaching in the context of GenAI is to ensure students realise that if their learning and their submissions rely on this technology, they will not develop the critical skills they need to recognise its limitations.
About the Author
Neville Morley is a Professor of Ancient History at the University of Exeter. His research involves ancient social and economic history, the influence of ancient texts on the modern world, and theoretical and methodical approaches to ancient history. Prior to joining the University of Exeter, Neville spent over twenty years lecturing at the University of Bristol .
Neville has written widely on the topic of ancient history and contributed his knowledge of ancient history to many popular media outlets, including the BBC’s In Our Time radio podcast. Neville is currently working on his newest monograph What Thucydides Knew for Princeton University Press, where he considers Thucydides as a guide to modern politics.
AI, History and Historians
The Royal Historical Society is currently hosting events and blog posts that consider the contribution of GenAI to History teaching and research in Higher Education and in related sectors such as archives amd heritage.
Current blogs in the series may be found here.
In July 2024, the Society held a first event, a panel discussion on ‘AI, History and Historians‘, bringing together AI specialists and historians, and which is available to watch or listen to again. Further posts and events on the theme of GenAI and History will be announced on the Society’s website.
HEADER IMAGE: iStock – Credit: ReotPixel | First Image: iStock – Credit: Urupong | Second Image: iStock – Credit: AliseFox