Scaffolding Coding Competence: Teaching Coding and AI with Critical Intent (CPD Case Study)
Dr Fergus J Chadwick (School of Maths and Statistics)
[email protected]
1.What motivated you to use AI in this module, and what goals or challenges were you aiming to address?
With my colleague Alison Johnston, I have been adapting our approach to AI in our “Software for Data Analysis” module (MT5763). The module is focused on teaching students how to use programming languages to carry out all stages of data analysis – from scraping data off the internet and cleaning it to statistically modelling data and visualising the results. AI is amazingly adept at coding! The way programming languages are structured, and the large amount of freely accessible coding examples mean that AI programs like ChatGPT can be very effective at coding. Indeed, many professional coders now use AI to code faster. However, like with any AI output, it is essential to be able to critically appraise the output, and this can only be done by understanding the fundamentals of coding first. As such, we did not want our students starting out by only using AI tools. (the “monkey with a machine gun” approach). But equally, we did not want to pretend that AI tools are not a useful and important part of their training (the “abstinence only” approach). Our goal was then how to help students learn the fundamentals of coding and then exposing them to augmenting their skills with AI in a critical and thoughtful way.
2. How did you design or adapt the assessment, and how did you prepare students for using AI appropriately?
We had two prongs to adapting the assessment. The first was to start assessing coding more in class and in person where the students have less opportunity to use AI during the assessment. The second was to add in questions on using AI directly, including examples where AI prompts or output are poor and asking the students to critique them. In both cases, the material was covered in lectures and in the future, this will be incorporated into class notes.
3. What challenges did you encounter, and how did you address them?
Some of the main challenges in taking this approach were administrative. AI is advancing so rapidly that assessment techniques that have worked previously can become outmoded and ineffective very quickly (and somewhat unpredictably). Simultaneously, we are bound by (important and necessary) checks and timetables for changing assessments. As a result, updating assessments is an inevitable compromise between what might be optimal and what can be implemented on relatively short notice.
4. What benefits did you see for students and for your own teaching practice?
I think for me and the students the biggest benefit is realising that the core skill we are teaching is problem solving. The specific details of any given piece of coding are largely irrelevant, but the process of findings bugs is the same whether we are handwriting equations, typing our own code, or analysing the output of ChatGPT. These problem-solving skills are what makes graduates from St Andrews (particularly in maths and statistics) so attractive to employers. From a teaching practice point of view, it has been useful to challenge some of our teaching paradigms about how fixed some of the content is. We have had to take a much more agile and flexible approach to delivering the material than previously.
5. How did you evaluate the usefulness of this assessment to ensure that it reflected the desirable learning outcomes?
We are still waiting for all the assessments for this module to be completed but we will be looking to see that students have genuinely engaged with the AI component of the module in a critical way while still performing well on the non-AI parts. We want to see that both parts of the assessment allow a student to distinguish themselves. Finally, we hope to see the ideas introduced in this module carried through to the students’ subsequent work on their degree programs, particularly in their dissertations.
6. What would you do differently next time, and what advice would you give to colleagues?
Next year we are hoping to use AI-free environments to greater effect through mechanisms like class tests with AI-blocking on the computers. This year, we unfortunately were not quick enough off the mark to implement these changes (this is our first year teaching the module) so perhaps did not realise the level of admin coordination that is required to make these changes. My advice to colleagues on this specifically is to act as early as possible and be aware that even then it will be necessary to compromise for the sake of pragmatism. More generally, I encourage colleagues to engage with AI as a tool that students need to be taught to use and respect but it is not a panacea. The core problem-solving skills that we have always focused on teaching are still essential in the days of AI. Perhaps even more so!