With over 145 registrants the first UK Symposium of 2023 was an engaging discussion on the impact of AI tools on teaching, learning and assessment. The general consensus being these tools aren’t new - indeed we can look as far back as the arrival of the calculator - and they aren’t going away, so it’s all about understanding when and how it's appropriate to use them, education for both staff and students, redefining what we actually mean by cheating, and whilst being mindful of ever increasing workloads, re-think quality over quantity of assessments.
Read the full discussion transcript from our expert panellists - including answers to the many questions from the highly engaged delegates - from the 28 February session.
- Professor Helen Laville, Provost, Kingston University
- Dr Irene Glendinning, Academic Integrity Lead, Coventry University Group, QAA Academic Integrity Advisory Group
- Professor Alison Truelove, Associate Professor in Critical Practice, Lead Academic Tutor and Director of the Centre for Innovation in Business Education, University of Exeter
- Chair: Professor Judyth Sachs, former Deputy Vice Chancellor & Provost at Macquarie University, PVC Learning and Teaching at the University of Sydney, and now Chief Academic Officer at Studiosity.
>> Watch the full recording, here.
Prof Judyth Sachs: Good morning. My name's Judyth Sachs. I'm the Chief Academic Officer for Studiosity. I want to acknowledge that I am hosting this online conversation from the lands of the Gadigal people of the Eora Nation. I am located in the suburb of Balmain in the city of Sydney. I also pay my respects to elders past, present and emerging and celebrate the diversity of Aboriginal peoples and their ongoing cultures and connections to lands and waters of New South Wales and elsewhere in Australia.
I welcome our UK colleagues and those from other countries. It's good to have you participate. Today is our second symposium for 2023. The week before last we had one hosted again by me and we had participants from New Zealand, Canada, the US and other European countries. We had 1634 people registered and about 1200 participated. Today we have, we have over 238 registered and the numbers are going up now. So just let me make a couple of observations and then I'm going to ask the panel members to introduce themselves and to indicate why we've asked them to be the spokespeople in today's symposia, your area of expertise.
So a couple of observations. Every day there's some mention of AI in general and ChatGPT in the media, and media feeds and social media sites. The first response four weeks ago was that of moral panic. The end of the universities as we know them was imminent -according to the moral panic. Some of the reactive hit has now subsided, and now there's more informed and reflective debate on what this new technology means. There have been a number of webinars and there are an increasing number of interesting discussions on various websites. Discussions of the ChatGPT are around technology, around technology dimensions of those of learning and teaching assessment.
Our focus today is on learning and teaching and to consider what new opportunities may emerge and what we need to do to be watchful and responsive. So the seminar today is introduction 5 minutes, that's nearly gone. I'll then ask each of the members of the panel questions that relate to their expertise and experience. And then questions will be taken from the audience. So please, if you have questions, put it in the chat and I will try to synthesise them and get the discussion going, but let's let the panel introduce themselves and Irene, may I ask you to start, please?
"let's let the panel introduce themselves"
Dr Irene Glendinning: Thank you very much. And thank you very much for inviting me on the panel this morning. I work at Coventry University in the UK and my role is Academic Integrity Lead for the whole of the Country University group. And so I coordinate all the policies. My expertise is in policy and I've been doing quite a bit of research into how we should design both policies and guidance relating to the use of artificial intelligence to all of our students. I'll say something more about later on in the panel.
Prof Alison Truelove: Good morning, everybody. So I'm joining you from the University of Exeter. I'm an Associate Professor in the business school there and Director of the Centre for Innovation in Business Education and as such, I oversee lots of work that we're doing at the moment around reviewing our assessments, not least in reaction to the arrival of these AI tools that we're going to be talking about today. And for me, my interest in this area is very much about seeing these tools as an opportunity. Yes, they are a threat. I think we're seeing in the news headlines even today, there's a headline around misconduct and the OIA putting out a report around the ways that we need to be looking at misconduct afresh because of these the emergence of these tools. And then yesterday, a headline around the IB, the International Baccalaureate, allowing the use of ChatGPT. For me, that's the wrong question. So I'm really interested in to interrogate some of these responses. So thank you very much for inviting me and look forward to discussing some of these issues further.
Prof Helen Laville: Thank you. I'm Helen Laville, and I'm the Provost at Kingston University in London and I think that my interest in this topic, or the reason I've been invited, is that it's very possible to have some very, very interesting pedagogic discussions and design fantastic, great practice that's really innovative and supports students, and then at some point you have to translate that into a framework of being able to put it into action, monetarist, if I can use that word and make sure it works for everybody across an institution which covers a slew of different disciplines, approaches different kinds of assessments, student support versus academic workload and time and ability to manage that. So I think I'm really interested in that term pedagogic and intellectual discussion around it. But then at some point there will have to be a discussion about how do you then implement this across a university infrastructure, bureaucracy, administration. And that's sort of devil in the detail bits that often trips us up even when we have the best intentions. So I think that's kind of like my interest as well.
Prof Judyth Sachs [00:07:13] Thank you very much. My training was in anthropology and in ethnography in particular, and there are three questions that ethnographers ask, what's happening here? What's really happening? And what does it mean? So in the context of this conversation, today the first one, what's happening at the moment for this to be an issue in universities? And I don't mind who wants to start that off. Helen?
"what's happening at the moment for this to be an issue in universities?"
Prof Helen Laville: I knew you were going to call on me, I was just looking for my unmute because I thought having said that I'm looking for that sense of how we implement it. I think we have, we've sort of known that this was coming and there have been discussions and interesting things start to talk, but it's been a bit speculative and it's only recently that we've all been allowed to have a play with this and get
onto ChatGPT, and I know there are others out there as well, but it's really something that feels like it did catch us a little bit unawares and we all playing a little bit of catch up. So I think at university level, like other universities where reviewing what we say, there is a, I like your phrase moral panic, because I think there is a moral panic in the press and in the policy that is often based on the people who write these things think that universities have not moved on since they were at university. So they have a very strong picture that everything is a written assessment that can be written by artificial and test, and that's how universities work. And so this will be devastating. And in fact, when you look at it, you know, the strides universities have made in the last 20 years to more authentic assessments and different varieties of assessments is tremendous. So there is already a lot of assessment that will not be touched by this. But in terms of what universities are doing, we're looking at our policies, we're looking at our guidance, we're looking at student support around assessment, things that we have been looking at anyway and then was accelerated through coronavirus and the changes we had to make there. So we're looking at the policy, but that can be quite dry and hard to interpret. And you know, how many of your students really read some of these policies? They're not really written in student friendly language often, and even if they read them, you know, they're quite difficult to interpret. So we're looking at what sort of student support we put in, because I think with with with these things like ChatGPT and the others, more than anything, we need to be able to work examples into our teaching practice so that students can really see what we're talking about. So there's a connection between discussions we have of policy, practice, good ideas, guidelines, and actually making sure they don't just get uploaded somewhere, but they are part of how we support students and prepare them for assessment.
Prof Judyth Sachs: And that they're living documents so policy doesn't get ossified. An audience member has made a comment suggesting that some policy makers develop policy based on their experiences 20 to 30 years ago. Yet that sense of renewal must happen all the time.
Prof Helen Laville: Yeah and that sense of I mean, again, some of our policies around extensions and mitigating circumstances and so on often are based on the idea of small assessment - sorry, end assessments at the end of the big piece of work that you can manage. And they haven't actually picked up. They haven't kept pace with some of the continual assessment that we do, the small early stakes assessment, the way we sometimes use assessment as a proxy for engagements. I don't think our policies have really kept up with that. So it is part of the bigger discussion about how we manage assessment and what we assess for, you know, is assessment of learning or assessment for learning. You know, how do we embed it a little bit more? So I think it's it's part of that bigger review.
Prof Alison Truelove: Thank you. Yeah. So I think part of this is about redefining constantly what we mean by cheating. I think our perception of cheating might be different from every student perception of cheating. I think when we've got students coming in from different countries around the world who previously studied either pre 18 or indeed coming to us at master's level, having studied undergraduate level, policies differ, and perceptions of what constitutes cheating differ hugely around the world. I think the emergence of some of these tools that are causing such panic really is refining our idea of that. 
Because in the past, you know, we've accepted that students use Google. We might encourage them to use Google Scholar or go via our university library searches and databases instead. But essentially, you know, how different is asking ChatGPT for a summary of a topic, to turning to a friend or turning to a parent, or turning to one of their peers. So we've got to really be clear with students what we mean by cheating, by what we mean by misconduct. And that ties in with how we create that policy, but also how we communicate that policy. But it's also about adapting our own intended learning outcomes. I mean, we all call them something slightly different, I think but in the end, what is a degree program? What is a module? It's about teaching something, these days very much more on the skills and the competencies that students are developing, not just that knowledge. You know, our students will frequently say to us, we have Google, why do we need to remember things? Why do we need to be tested on our knowledge? And of course, they they do need that as a basis on which to then extend their learning. But we are assessing different things now, I think, than we were assessing perhaps 20 or 30 years ago. And there's much more of a focus on employability as well. So we're thinking, particularly in business schools, around building those authentic assessments, really building up those competencies. And I think the emergence of these AI tools really are making us think much more carefully about how we develop those digital competencies, and that isn't just in students either. That's in staff as well, and how we support them to build up an understanding of how to use these tools effectively and assess students appropriately.
Dr Irene Glendinning: Yes, I'm going to broaden it slightly whilst going back to about last March, I set up a working group in my institution to look into primarily, we were looking paraphrasing tool. We were looking at paraphrasing tools, grammar checkers, translation tools, and also we were looking at essay bots. That's where we started. So we've been exploring that for almost a year now. And these tools are not new. They've been with us a very long time. We use them from day to day, we all use them. 
They're going to be embedded within software that we use all day, every day very soon. Some of these have been embedded within these tools for a very long time. We've got used to using them they are fine. But the advent of these content generators, it's not just X there's other things as well, has changed the need to respond. So what I've been doing is to look at with my team, with my colleagues, we've been, best of all, expanding our own knowledge on these, but also talking to colleagues. And that's really important part of the process so that we can then understand all the different perspectives on when it's good to use these tools, when it's useful to use these tools, and when it's not appropriate to use these tools. I think that's really what we need to focus on. We can't ban them and I think everyone agrees with that now. It would be impossible to ban them even if we wanted to. But we need to think in a nuanced way about what would be appropriate uses of these tools and when not, and then communicate that both to staff and to students. Make sure people understand that when they and they follow the rules once we decide what they are. But that we're flexible in terms of what needs to happen because these tools are still evolving and they're not going to go away. They're going to be there forever now. So we need to think about and take advantage of what they're good at, but make sure that we control the use where we can and make all the rules very clear to everyone, so that that's really what I've been working on.
Prof Judyth Sachs: So I'm getting the sense that the common thread here is that in fact, it's about clarity, real clarity of expectations. It's around reframing assessment, and it's making assessment purposeful. But it's also about accepting that this technology will continue to develop. Have I have I got it right or am I sort of off?
Dr Irene Glendinning: You have. I think that's, I think you've summed that up quite well. And because the other aspects of what the two panellists talked about designing, designing pedagogy, designing assessments, and I think we are already half way there. A lot of a lot of the assessment redesign and the dialogue around assessment to cope with contract cheating is very much in line with what we need to do for these tools. So we need to keep moving in the same direction. I think I think the pandemic helped us a lot because it made us re-think, what assessment is for, and what it needs to look like. And all of that work that we've done for that is contributing directly to what we need to continue to do it regarding the AI tools. I don't think we need to panic, we need to just take a measured approach and keep, keep moving in the same direction.
Prof Helen Laville: Yeah. Sorry - I'm looking at the chat and the conversation there and I think, you know, again there is that, I don't think anyone, we've all, no one has so far has put we should just ban it. We all recognise that that's just first of all an arms race we will lose. We'll constantly have to keep upping our investment in detection, it just won't work. And secondly, it doesn't reflect the reality of the world and of employments where people will be using these. So I think it is much more about a thoughtful response to it and thinking about what students get out of it. It's really interesting that I think that some of the coverage is focussed on the idea of it's artificial writing and when actually when you look at it, the writing style is quite bland, it's quite unimaginative. I think the writing itself is that, it's very clever for AI obviously, but I think the bit that is interesting to me sometimes is how students struggle with structure, how they get going with things. So I think somebody has put an interesting comment in the in the chat about sometimes this is a sort of starting point. It's a framework for students. It lets them take something that helps them think about their structure, which is something I think students struggle with more than writing often and then they take it from there. So what we're really talking about is the starting point is different. In the same way that Wikipedia, Google, moved the starting point from go into the library with a blank piece of paper. This is moving the starting point for that piece of work further, further along. And I think what we need to do in assessments is to be able to enable students to be honest about that and to tell us, you know what was your starting point? And where did you change from that? So so that they can reflect on that piece that 'I started here. I started with a structure and some general generic ideas, and I read it and this is bit I really didn't think was right or this is the bit that I wanted to investigate more'. So I think allowing students to have that discussion as part of the assessment in a way that art school pedagogy has always done, art school pedagogy has always been, it's about the process and it's about your ability to reflect on the process. So I think that's the bit we need to start thinking about. It's not assessments, not about the product at the end of it necessarily. It's about that discussion, that debate, that ability to reflect on how you got to where you got, where did you start from, and then what did you do to move things, to add the human intelligence to the artificial intelligence.
Prof Alison Truelove: Yeah, I think we can see this hand in hand as well as the other challenges that we're all facing in higher education at the moment. And that around student attendance and engagement. So I think actually this is, this whole debate is helping us with that because it's forcing us as educators to develop the way that not only we assess but how we teach and how we embed that assessment and link that assessment with everything that we do in the classroom. I have an example of I don't know if now's the appropriate moment Judyth, to just tell you a little bit about an example of a module that I designed with some colleagues at Exeter a couple of years ago when we, we were really trying to think through not only that the module is called Digital Technologies and the Future of Work. So it was very much about how we integrate and enhance our use of digital tools and how that they will impact us all as we move into different jobs throughout our lives. And we wanted to really address this issue of attendance and engagement. So we built an assessment where it was portfolio based and students were only able to do well in that assessment if they had actively engaged throughout that whole module. So we had online forums, discussion forums, which we know students were not using very much at all in many other modules, but we directly linked it to the assessment. So students were required every week to contribute to those discussion forums. They had to then have a portfolio of evidence that they would submit as part of that end assessment, to prove that they had engaged in discussion with their peers. These are things that actually we we should always be having students do. We would hope that students are doing throughout any of their courses, but by linking it to the assessment, we were kind of forcing the issue and making it difficult for them to avoid that. It helped with the attendance at class, even if they weren't actually physically attending classes and some of these were online classes, no doubt, obviously because of through the pandemic. But it meant that students had to engage and it meant that the assessment was much more personalised. And I think that's the other way that we get around use of these tools is the assessments need to be personalised, tailored, unique to the individual students and help us to see that process of learning. So it's difficult sometimes to navigate that with students who are not used to this way of teaching, but I think it's a really useful way of thinking about the challenges that we face when there are these tools available for students in creating their assessments.
"the other way that we get around use of these tools is the assessments need to be personalised, tailored, unique to the individual students and help us to see that process of learning."
Prof Judyth Sachs: Do either of the other speakers want to respond to Alison's point? And I think it's something that comes up for me, but I'll wait for the others to make a comment or an observation.
Prof Helen Laville: I was just going to suggest that I absolutely agree with that kind of continuous using assessment is itinerous breadcrumbing is really ideal. All I was saying about how we then translate that into policy, where we have done that, we have ended up with a massive backlog of mitigating circumstances. The students get very, very obsessed about every single piece of that assessment counting, and everything counts so we were overwhelmed with mitigating circumstances claim for things that when you added it up was less than 1% of their overall degree and we were forcing them to fill in forms and get doctor's notes and things like that. So there is a sort of policy translation, that's the bit I'm kind of interested in, is how you take something that is pedagogically works and then translate it into policies. Because I think the idea, as we've said, is absolutely that's the right kind of assessment. But I do think there's something about helping students understand assessment so that it's not such the high stakes, stressful, continuously stressful experience. That they're able to see that there is some space in there to to fail at something or to not do so well at something and to see it all as a big piece, because I do think there is a sort of, it's all that connectedness that then connects with student mental wellbeing and that constant feeling of being under pressure. So I agree it's the right approach, but we need to think through how we embedded through our policies, student support and their understanding of what's being asked of them.
Dr Irene Glendinning:I agree with all that's been said, but there are other ways of assessing as well I think that are useful and helpful. And having a, having the student and the presence of the student where possible, either online or in person, is another way to make sure that knowing your students all of that is important part. Asking them to do something active rather than something just go away and create me an essay on this topic that's already out there. So there are, it's very important that every person setting the piece of assessment and it's kind of what, how easy it is for a student to gain access to that information, for example. Making sure it's applied. We know at the moment that these tools can't write critique. They're not very good at writing critical appraisal, and they're very much fact based. Making sure that it's personalised to either the context of the institution, context of the student. Those are the things that artificial intelligences can't do very well. They have to understand the context there. And also we need to understand that essay mills will also be using these tools so, and in fact indeed they are already because it's cheaper to employ an artificial intelligence and to employ a person to be a ghost-writer. So you need to understand the kind of the full range of different approaches. But I mean, we one of the ways we've been kind of focussed so far on this is assuming that because these tools are out there and because these opportunities are out there, students will all cheat. And I think we have to be realistic and understand that the majority of students are honest and that they're there to learn. They pay a lot of money to us to be taught and to engage in the learning process. And the honest students, as long as we give them good guidance and support, the majority of students will, are there to learn and will engage with their learning and it's the students who are minded to change that will continue to cheat. And it's my hope and it's my expectation that we won't have a massive rise in the number of students who are breaching integrity as a result of these tools. And the students who are minded that way will will just pick on the tool that are available. And so but we need to get the message out to the students that if they are minded to breach integrity we we can and we will be able to detect it. So that's a whole new topic that we haven't talked about yet.
Prof Judyth Sachs: An audience member has put together an interesting point about critical thinking to me is crucial in all of this: We need students in undergraduate and at A-level to be able to demonstrate critical thinking and communicate this effectively. The secondary school curriculum is very much geared to rote learning rather than critical thinking. So how do we encourage and nurture original, critical thinking and engagement in students?
"So how do we encourage and nurture original, critical thinking and engagement in students?"
Dr Irene Glendinning: Can I just start by saying I completely agree with that point. And in some ways the threat to education is more about the secondary level than it is about universities, because I think we're already on top of this this subject. Whereas our schools haven't even started to think about these problems. So I think we do need to engage with our, if you like, our feeder schools with secondary education within within our countries, within our regions, and help them to come to terms with this. Because if the students are using these tools to replace their own writing at a younger age they're going to be devoid of these writing skills that we want them to develop. So it's really important. It's a bit like going back 30, 40 years when calculators came in. It didn't take away the need for students to be able to do sums, you know, being able to do arithmetic. Because if you can't do mental arithmetic and can't function in your general life it's important that student have it. In the same way, they must learn to write at a younger age and not rely on tools too much. That's my view. Once you've learnt to be able to write and read and understand what the quality of the outputs are from these tools then you can use them in an ethical, intelligent way.
Prof Helen Laville: Yeah, I think, I think it's I mean, we all know the answers to these. There are lots and lots of imaginative, creative, interesting answers that people will come up with across the university sector. And again, I'm sorry to keep going back to the administration, the policy part of it. But a big part of it is just workload. People having the time to do these things, moving away from things like, you know, the reason schools do this again, I think we're stuck on the idea that people who look back and say, well, I passed A-levels, therefore they must be the best measure of brilliance. But I think often it's just about workload, timetable, ability to manage it. So I think there is a bigger question there as well about that probably addresses the fact that we are consistently over assessing. And a colleague of mine did, at the University of Leeds did a marvellous piece on this showing how much time we were spending, assessing students, marking students, administering the marks and questioning whether that was really a good use of everybody's time and someone in the chat posts, chat comment, as mentioned, you know, do we want to move away from that sort of very modular, very every single learning outcome assessed in a specific way? Could we reduce assessment so that we can be more creative, more clever and more in-depth about, as you say, Judyth that critical thinking piece, which involves a lot more time, I think. And as somebody again, has put in the chat, you know, we can use ChatGPT i's not a one way street. We can find academics able to use it to give feedback. There are other things we can do with it, but we want to have, we want to have the better discussion around critical thinking. And to do that, I think we need more time to devote to that. And in order to do that, I think we need a better idea of, you know, let's have less assessment, but at a higher quality and then we're really having more engagement and more discussions about that, because the realistic aspect of it is people are very tired, very overworked, and the volume of assessment is overwhelming. And of course I've done it myself, we default to I will just do what I did last year because it's okay, it's fine, I got by with that. And I think we have to have a bigger discussion about how we manage assessment in a way that is manageable for people. And then you get to this kind of quality assessment and quality feedback that students want and deserve.
Prof Alison Truelove: I was just going to come in on the critical thinking aspect of this because I think actually we can harness these tools like ChatGPT to really support us with that, to really get students to interrogate the things that they are reading and scrutinise them and actually use the outputs of ChatGPT within the classroom, you know, generate various different responses to particular prompts. I think the prompt writing is an interesting issue in itself. I think actually part of our job as educators, if students are going to be using these tools throughout their education and their working lives, is to teach them how to write good prompts. I think we've all played around with these tools and realised that the first question we put in does not produce a very good answer. So maybe we could have this as an active activity within the classroom and actually get those bits of text generated and then talk to students about why they may not be good. You know, what is the material that these chat bots are drawing on? How high quality is that? So when the piece about IB students being allowed to use ChatGPT and quoting as if it's any other source ignores the fact that actually ChatGPT is not acknowledging the sources that it is using. So there's something else called perplexity, isn't it? There's another AI tool out there that we can access at the moment which does provide citations, which at least goes a little bit of a step further there. But I think we've really got to get students thinking about everything they're encountering. And it isn't just text here, it's images, it's films, it's anything else that they might be encountering as part of their studies. Asking the right questions. You know, those classic who, what, why, where, how questions, who produced it, Why did they produce it? Who is that? What is the intended audience? Where is that potential bias? That's how we develop their critical thinking skills. So I really think there is an opportunity here in using these tools to get students to really be thinking seriously about how they're accessing information and the quality of that information and whether it's reliable or not.
Prof Judyth Sachs: Can I just redirect our conversation a little bit? An audience member makes the comment: I think we're failing to address cultural issues. Recent expansion international student recruitment has revealed deep differences in students' prior experiences and understanding of assessment offences, willingness to accept assessment offences, and even with the clearest evidence, shock that we really will see through the policy, especially on serial serious offences, and strong financial and familial pressures to succeed. 
This is not true of all nationalities, but there are some patterns that we tend not to touch on for fear of discrimination. If those populations are making decisions from a very different basis, though, I suspect we are hugely and underdiagnosing UK students' use of mills etc. as the language differences are not so stark. So what's what's the point? I mean, the issue is we're seeing the student body as sort of homogeneous, but it's not. It's very diverse. It's very varied. International students pay a huge cost financially, but also in terms of their social relationships. So I guess the question is how do we manage the differing expectations of students and how do we support students to be successful rather than fall into a trap of misunderstanding of the norms and requirements of a UK university?
"how do we manage the differing expectations of students and how do we support students to be successful rather than fall into a trap of misunderstanding of the norms and requirements of a UK university?"
Dr Irene Glendinning: Can I come in at that point. I think this is a really, really important question, and I don't think we do enough to, to introduce our international students to life in the UK. We are doing our best - or we think we are. we can be all that we can do. I think there's far too much assumptions, particularly when a student comes in at master's level, that the students will already have all the skills and that they should have the same kind of values and motivations that we do. And I think we need more to we need to do more to bring students into our community and make them welcome. We had an example came up recently by one of my colleagues talking to a student who had been facing allegations of academic misconduct, and the student haven't been using any of the services within the institution at all. Because he didn't think it applied to him and how we could kind of try and break down those barriers to make sure the students are not just aware of what services are out there, but also that they do apply to them and they are very useful. So in my institution, we recently introduced what we call success coaches, where the students can go to a mentor and talk to that mentor about their problem. And very quickly mentors contacted me and said, Irene we need to know more about the academic integrity policy because that's the kind of questions that we're getting. So it's kind of finding ways to join up all the different parts of the university and the parts of the services that we provide and make sure that people can signpost correctly and advise students on the different things that they would need to know while they are students here. And I think there's a lot of a lot of presumptions that it's not my job to teach students about writing. It's not my job to teach them about paraphrasing and all of those things. There are a lot of skills that students need to apply when they arrive here, and that applies to UK students as well as international students. And I don't think we are putting enough effort into that kind of education piece. And doing it as a joined up process and I think that's really important that it is seen as as part of the curriculum and that it's orchestrated perhaps by the course director or the course director says right, we have a period of time here where the students need to learn about writing, to develop their skills, but also continue to develop their skills throughout the student journey. It's not something that just once when we write something that needs to continue to develop. As is critical thinking, critical writing. A lot of the students that have arrived have only ever done exams as their assessment before they arrived. They haven't really had to do academic writing the way that we expect them to do.
Prof Judyth Sachs: Irene can I interrupt and give the other two a chance to respond to some of the things that you've been saying. And I think what you've been saying is absolutely on on, on the spot. But I've got to be fair.
Prof Helen Laville: I think that I would completely agree with that. And I think what I can't remember the exact title for this session, but I think I like the fact that it was called something like a student centred perspective. So, you know, can we do stop thinking about these issues from the point of view of, oh no, how do we manage this as a university? How do I manage it as a teacher to, to what does this change about the students' way of approaching assessments? And I think that point about international students, but also some of our first in family, first generation students to go to university, about how they see assessment is really, really critical to this discussion, because if you're an academic, you see assessment every single year for many years now, you've been taking them in, marking them, giving feedback over and over again. It becomes a commonplace thing to you. And I think that we then forget, of course, sometimes that from the student perspective, they're at university for three years, one year for their master's, whatever it might be. This is absolutely critical and crucial to them and their stress point the emotional, social intellectual capital that is invested in that piece of work is enormous, is enormous. And so when we understand it from that point of view, and I know many people on these calls will have been in misconduct committees or appeals or anything like that. And this idea that you can sort of talk about cheating is just as basic dishonesty 'I knew it was wrong and I did it'. So few cases are actually about that. When you actually start to unpick them it's students who are under tremendous pressure, do not, as you've said, don't understand where to go for help, don't necessarily always understand what cheating means. You know, they may think it's a form of compliments to be repeating certain academics words back to them, but slightly different. So it's about all of those things. And again, I think that student centeredness of, okay, how what are we doing assessment for? Is it to satisfy professional bodies? Is it to give people a school that employers will understand or is it to help students feel that they have learnt something? And for them to be able to articulate what they have learnt, you know, what's their learning gain is their assessment geared towards them understanding the value of what they've done. But I think understanding the pressure that students are under and why they take the routes they do is going to be really important. I mean, it's interesting, as I said, about GPT being a starting point. You can see giving all students a answer to something and asking them all to critique it or to talk about the bit that they are interested in sets as a nice common starting point. Instead of them feeling they have to hide the starting points or not acknowledge that they are at different starting points. So I can see how it could be a really useful tool in that way.
Prof Alison Truelove: Yeah, I completely agree, Helen, with your point around most students. Actually, not intentionally cheating. Most of it is panic. Running out of time, having troubles with the language. Most students feel a real sense of injustice around if students around them are cheating. They want us to really crack down because they think it's unfair on themselves when they put the work in. So most students, I think, are ethical. Most students, I think, want to do the right thing. And it is incumbent on us you're right absolutely Irene, we need to put so much more effort and I think we are doing a lot better around induction. At Exeter we've built pre-arrival courses that students are asked to go through before they even arrive on campus to try and kind of familiarise them with the ways that we will be teaching them, the ways that they'll be expected to study. We've designed a framework called Principles of Professional Practice, which includes being ethical. It means includes being collaborative, respectful, responsible, compassionate with others engaged in your classroom. I think these are the kinds of things that we're going to need to put a lot more emphasis on. And I think it's a particularly a challenge for those of us with very large cohorts for all the reasons that we've already touched on here, that we have to have that peer to peer support in place as well. So one of you mentioned earlier about having additional people on campus who are dedicated to the student experience. Mentors that will hold students hands if they feel that they need a little bit of help. We brought in some recent graduates for us because we were finding that students weren't necessarily going to their educators. They weren't going to the full time members of staff because it felt too intimidating for them. So we've got recent graduates who are now working with us because they just seem much more accessible. They sit, they have regular drop ins. It's very easy for students to go to them and access that support. So I think it's about trying to see the good in students. It's not always seeing them as is being able to cheat the system. Some of them inevitably will be. But I think we've got to see the good there. And one final point about international students, of course, is that we know many of them use translation tools. So what are we assessing here? Are we assessing their ideas or are we assessing their writing? We've got to think about what those intended learning outcomes are again, for each course that we're offering, because if we're not directly assessing their ability to write, should it matter to us? Perhaps if they've written a piece of work in their own language, translated it, maybe run it through ChatGPT to polish that that writing. If the core of it that we're assessing is still their ideas, should it matter that they've had help because if that again to reflect on my point earlier is that any different from getting a friend or parents to read their work and to maybe do some proofreading. You know, it's a sliding scale, isn't it? So what again, what we find acceptable or what we don't.
Prof Judyth Sachs : There's a comment here from an audience member: second and third language English students are more likely to use AI tools for cognitive offloading. How do we address this without stigma? And Irene, I'm going to throw that one at you.
"second and third language English students are more likely to use AI tools for cognitive offloading. How do we address this without stigma?:
Dr Irene Glendinning: Yes, absolutely. And the difficulty here is how we how we decide what this kind of grey area is and what - if we're going to allow students to use this tools, which I a strongly advocate we do, then we need to set some boundaries for what is acceptable practice and what isn't. And that's going to be the difficult part, is to define what that grey area is and define where if there is a reply, where that lies. And I think these tools are very valuable. They will be very useful. I've had as part of my working group, we've had a member of our English reference services, part of our team, and that's been really, really instructive because he's been helping us to understand how how they see these tools as being valuable to students. And we wouldn't want to stop students using grammar checkers because it helps to increase the value of their work. What we wouldn't want students to use is to to deceive, to use these tools for cheating purposes. So it's getting the message over to the students about what is ethical use of the tools and what isn't. And that's the heard bit. We haven't defined that fine detail of the policy yet, but what we are saying we would like to do, I think this is coming out of quite a few of the kind of policy developments I've seen, is that we'd like to ask students to acknowledge when they are using tools and perhaps the cracks of the work they've done so that they can demonstrate the challenge about it, that they can demonstrate which bits of it were their own work, if you like, where they where they got help from the tools, but also not use the tools to be seen, such as in agreement with translation tool, so not take a plagiarised essay in another language and then translate it into English and hand that in, those are the kind of no go areas. But we need to define rules. We need to define for students why that's wrong. And I if I can pick up on something that Alison said about it's okay to use translation tool, I completely agree. Translation tools can be very useful for both reading and writing. We wouldn't want to ban them - I use them all the time. But what we wouldn't want the students to do is to use them in a way that they are taking the piece of work and using the translation tool so it reduces the similarity percentage, for example, to this kind of plagiarism and so on. So it's that kind of message we need to go to students and that really needs to go into the policy. But what we, the way that we do that is we ask students to declare when they use the tools and how they use them. And in that way, we've got some kind of control and honesty over the use of the tools.
Prof Judyth Sachs: It seems to me that there's a whole lot of consensus about things and one of them is it's about learning. And it's about support. And it's not just students learning. It's about academics being provided with the skills to be able to use it. So can we just investigate that whole assessment, for want of a better word let me call it the assessment ecosystem, to to to help some of our academic colleagues who have been doing the same pieces of assessment for the last 15 years and are really resistant to changing it because they are they are absolutely convinced that these questions demonstrate something. I don't know what they demonstrate, but they clearly demonstrate something. So how do we create that learning ecosystem for everyone? And so everybody has skin in the game around assessment, but also around issues of integrity, where to attribute and the whole the whole quality of the learning experience.
"how do we create that learning ecosystem for everyone? And so everybody has skin in the game around assessment, but also around issues of integrity, where to attribute and the whole the whole quality of the learning experience."
Prof Helen Laville: I think there's often data that you can use to back this up, and that's been really useful in talking to colleagues who know the unintended consequences of the thing that they're very committed to. We've been able to show them that. So we did some really great data working with my colleague who I see on the call, Avril from Man Met, when we looked at what was the implication of the kinds of assessment were having on awarding gaps. And what we found was the kind of assessments we were using in some modules were just year after year, resulting in an awarding gap to people who were BTEC students rather than A-level students. And that had implications in terms of their where those students backgrounds and from ethnic minorities were more likely to have BTECs and that implication, that idea that there is one assessment that is neutral to student background was something that we were able to challenge with academics who, you know, some of them were very horrified by these results. And we could show them that by changing the kind of assessment that they offered to students, they were getting what we think was you know, fairer results for students that we were assessing, as you say Judyth, assessing learning, not where they've come from or their competencies and their cultural backgrounds. And that was really helpful. So I think sometimes we do need to use that data dashboard and to sort of take that back to people and sort of say, look, this is this is what's happening, this is what's showing up in your data. And the same with we do a lot of work at Kingston about looking at pass at first attempt. And that's been really helpful because the data behind that is not just have you passed at first attempt, so are we helping students to pass the assessment, is it something that's doable? But it shows where we've got very, very high levels of students postponing the inevitable because they don't know how to tackle it sometimes. So some of those cases would be student illness, but you've got lots and lots. It was suggesting to us that the students were finding it really difficult to time manage around that assessment or so on. So I think seeing that whole picture, the ecosystem as you've said, and having the data that explains what happens. So not just looking at one assessment in isolation, but looking at the impact of that and showing students that students don't pass on first attempt, they're less likely to get good honours, they are then less likely to get good jobs. You know, they're less likely to stay with us. All of the implications from those things, putting it into that ecosystem and showing that back to academics is is very persuasive.
Prof Judyth Sachs: But is it also something about, you know, the great paradox is for many academics, learning is something that's alien to them. They've forgotten how to learn, and particularly in this digital age. So how many, how many academics - and I'm sort of being slightly facetious - but that sense of investing your limited time in having some training, having some support to improve the quality of your assessment and in fact improve your workload so that you're not you're not actually stretched in ways that we've we've all experienced. You know, teaching a first year psychology class of 900 students is pretty challenging.
Dr Irene Glendinning: Now, I was going to raise the issue about staff development because I think that's a really important part of it, we can't assume that all academic staff come in with the perfect teaching skills, perfect skills assessments. And I think the landscape has changed over the years while I've been in university and the kinds of assessments that I used to set down at that time, are not the assessments that we're setting now and we've moved the goalposts hugely on that. So I think a lot can be taught to academics, but they need to devote time and often they, these development sessions are optional and the busy academics who really need to have that kind of extra skill are not the ones to attending the sessions, but the ones that do I think they get a lot out of it, and I think it's moving the assessment story forward. But it also it affects the way you teach well as pedagogy as well as assessment, it's not just the way we assess. It's a way, it's a way we use the time with the students in the classroom and if you have 900 students and how you how you can find the time to assess those students and where have lots of different markets, how you triangulate the marking to make sure everyone, the marking is fair. It's a big, it's always a big challenge with margins of students.
Prof Alison Truelove: I think we can learn a lot aswell from our response to the pandemic. So in 2020, we all swung into action because we had to. We had no choice. So I saw I run weekly or ran weekly seminars on campus for colleagues to talk through issues around teaching and learning enhancements. Where we would get, I don't know, 20, 30 of people a week, that jumped to about 100 a week in that summer of 2020 because everybody knew they needed to be shaking up what they did. They knew we were heading into lockdown learning. They knew they were going to have to learn the tools of how to use Zoom, how to use teams and so on. So there was a motivation there. And I think we need to use carrots and sticks. We've got to have that motivation that is for them in the excitement of using new approaches to teaching and learning and how that can make teaching much more enjoyable, how it can help students see the value in attending rather than just thinking they're going to catch up through some online resources. But it's also got to come through policy. And we need support from leaders within institutions on this. We need support from outside of institutions in terms of those national policies and and and the nudges that we might get from our accreditation bodies, for example, we need to harness all of that. So there is that imperative, but there's also the attraction in engaging with that development and persuading staff that actually this is for the good of everybody. You're not just doing it for students, you're doing it for yourselves. And if you can get that right within your classroom, you can draw on your research, you can embed, you can make sure that your teaching is research inspired. You can bring in those live examples of the things that you're doing in that other side of your academic life when you're not face to face with students and create education, that's better for everybody.
Prof Judyth Sachs: There's a an interesting point made by an audience member: much of the conversation has revolved around generative AI's application to writing mapping, mapping it to proofreading services, but what about those domains where the output and the learning outcomes are analogous, for example, mathematics or computing sciences, where the output from a generative AI platform can supplant the complete learning outcome? I'm going to throw that one at you Irene.
"what about those domains where the output and the learning outcomes are analogous, for example, mathematics or computing sciences, where the output from a generative AI platform can supplant the complete learning outcome?"
Dr Irene Glendinning: Yes, because I'm from a computer science background anyway. Certainly I think computer science, computer programming is very much like reading, writing, arithmetic. You have to learn to do it yourself first before you can start to use the tools. Because these tools, yes, they're very good at generating code and very good at solving maths problems as well, and we will do that. The code doesn't necessarily work, but also the skills that you need to be a computer scientist are not just about writing code. They're understanding the problem itself in terms of code, you need to be able to maintain that code, you need to be able to modify that code. You can't do that unless you have the basic skills to start with. But you have to have the fundamental skills to do that. So it's important that students acquire those skills before they start making use of the AI tool. But there is no doubt that when they get it into the real world, they get out into industry, they will use these tools to generate code and therefore they need to learn to use it and understand what they're doing and understand their, you know, any flaws in the code generated by the tool as well. So they they need the underlying skills first. And so that certainly will be the policy that will be advocated in my institution in terms of computer science. But maths problems are another thing as well as they will do the working out for you as well. So students will be able to apply this and this is one of the reasons it's really important that we talk to our colleagues in secndary education and give them all the support that they need, because I think the approach to academic integrity at secondary level is a lot less mature than it is higher education for all the reasons that, you know, they are kind of tied up with the day job really and they don't really have time to do all othe other stuff. So I think it's more important that we work with them and help them at this point as well, kind of we just do that.
Prof Judyth Sachs: Much of the discussion has been about written text. But there's this there's nothing I mean, and we know the soft the soft skills that are required, the graduate outcomes, being able to work as part of a team have effective communication skills and others. So do you think that the the moral panic has been around the dominance of written text as opposed to sort of a much more rounded view of learning and the whole university experience? And why do you think this is the case? I'm going to throw this one at you Alison.
"do you think that the the moral panic has been around the dominance of written text as opposed to sort of a much more rounded view of learning and the whole university experience? And why do you think this is the case?"
Prof Alison Truelove: Yes, I think I think this is an indication of how early we are in discussing these things, isn't it? I think we're all doing some very quick learning around what these tools actually are, how they all work, and I'm not sure we all need to know how they will work in the background. But I think the implications for the whole education ecosystem is is really profound. But at the same time, as we've touched on this conversation throughout, none of this is really new. It's just the next iteration. It's the next toy that's come along, AI isn't new. We've all been using AI for many years, most of us, without even realising it. You know, when we when we ask Siri or Alexa to tell us something that's AI is generating an outputs. So ChatGPT isn't new in that respect. So I don't know, I mean, I think this is something that we are constantly I mean, this is what we do with education, isn't it? We're constantly flexing we're constantly adapting. That's part of what makes university education so exciting, and that's part of our responsibility as educators, when students come to us, regardless of the subject that they're studying, to be constantly adapting to the new environments that we're finding ourselves in. None of us expected the pandemic to come along. None of us expected the whole of the university system to be able to adapt so quickly to using Teams and Zoom and other types of tools. So for me, it's about adaptability. It's about being open to conversation. It's about constantly thinking how can we get better at everything that we do? But at the same time, we're exhausted. I think we need to acknowledge that, you know, the pandemic the last two or three years for everybody. We've got to be careful in terms of, you know, we've talked a lot about the need for us all to upskill, and we've got to support our colleagues, we've got to adapt to all of this. But it is time consuming. It isn't just about student cohorts being huge. It is also about that general environment that we're in and we need the support of each other to be able to do that, and working alongside students.
Prof Judyth Sachs: So you're talking about wellbeing, staff, wellbeing and student wellbeing. What piece of advice would you give to your Provost or your Vice-Chancellor about supporting staff and student well-being? Alison.
Prof Alison Truelove:I have to be quite careful with this one don't I?
Prof Judyth Sachs: So I think you're got a new Vice-Chancellor too.
Prof Alison Truelove: I think we are capable of doing everything we need to do. We just need the time and the resource to be able to do it well. And I think we do face challenges in higher education. You know, the way it's funded. This is a government decision about the way that universities are funded, which is having huge implications for our capability to adapt and be flexible because we are working within limited resource and time. So I think my challenge to a Provost or a VC would be please resource us properly, please resource education properly. We know how to make it brilliant. We know how to make it as good as it needs to be. We know how to adapt to these things, but we need the resources and the time and space to be able to do that.
Prof Judyth Sachs: Being a Provost Helen, and I was a Provost myself, so I've had these conversations - what would you expect your staff to do? What would you expect them to come?
Prof Helen Laville: They are doing what I expect them to do. They're behaving, you know, they're bringing forward brilliant ideas, creative ideas. They are critical thinkers and they are curious people. And despite what some of the press would have, they live in the real world. They have not been in an ivory tower for 20 years. So they're doing exactly what they should be doing, coming forward with some stuff. I think that the point about resources, it's well made. But as Alison has said, we are facing a fixed unit of resource that has lost its real value through inflation. It's been the same for many years now and everybody wants resource. And again, among again, Steve and my former colleagues talked about we've got a lot of agendas here, the awarding gap, student wellbeing, student support, tourism employability, before I get on to, you know, my researchers also what my support, my knowledge exchange people want more support and there's a limit. It's a it's a fixed amount of resource that we have. So we do have to get creative. And I think that's sometimes about giving people permission. So I've often found in conversations with academics, they've said, but you know, I'm not allowed to change anything and I need to look at those structures, whether that's true or not. And sometimes it is and I need to make them more flexible. Sometimes it isn't. It's the mythology that grows up in higher education about what you can and can't do. So I think that, you know, there's that sense of we need to make sure our systems empower academics, support academics, but academics sometimes do need to sort of understand that what I said about if you need to spend more time doing this, you need we need those things you need to stop doing. And you are over, some of them are over assessing, and it's like one really, really well thought through critical piece of assessment is better than what you're doing at the moment. And we do have to make those tough choices because the resource is finite. So but as I said, Judyth, they are doing exactly what I expect to, they have embraced the new challenge of ChatGPT and talked about how we how we how we address this as a learning community as a whole and how we use this to get over some of the silos that aren't helping staff all students at the moment.
Prof Judyth Sachs: And Irene, you're going to have a very quick last word.
Dr Irene Glendinning: It will have to be quick won't it! I think constant change is the problem that my colleagues are facing, really and it would be nice just to have a period of time when we're not changing, but I can't see it happening any time soon.
Prof Judyth Sachs: Thank you to the three of you for what has been a really wonderful discussion and to all of the people who were engaging in the chat. It was it was a great, great hour that I spent, I hope was a great hour for the three of you. And I wish you a wonderful day and I look forward to spring coming in your part of the world. And thank you again.
>> Watch the full session here
>> Watch the full session here
To see past sessions and register for upcoming sessions, please visit our Symposium series webpage.
