ICER 2022
Sun 7 - Wed 10 August 2022 Lugano, Switzerland
Mon 8 Aug 2022 11:30 - 12:00 at Aula Magna - Session 1: Programming Assignments Chair(s): Brett Becker

This article explores the natural language generation capabilities of large language models with application to the production of two types of learning resources common in programming courses. Using OpenAI Codex as the large language model, we create programming exercises (including sample solutions and test cases) and code explanations, assessing these qualitatively and quantitatively. Our results suggest that the majority of the automatically generated content is both novel and sensible, and in some cases ready to use as is. When creating exercises we find that it is remarkably easy to influence both the programming concepts and the contextual themes they contain, simply by supplying keywords as input to the model. Our analysis suggests that there is significant value in massive generative machine learning models as a tool for instructors, although there remains a need for some oversight to ensure the quality of the generated content before it is delivered to students. We further discuss the implications of OpenAI Codex and similar tools for introductory programming education and highlight future research streams that have the potential to improve the quality of the educational experience for both teachers and students alike.

Mon 8 Aug

Displayed time zone: Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna change

10:30 - 12:00
Session 1: Programming AssignmentsResearch Papers at Aula Magna
Chair(s): Brett Becker University College Dublin
10:30
30m
Paper
A Pair of ACES: An Analysis of Isomorphic Questions on an Elementary Computing Assessment
Research Papers
Miranda Parker San Diego State University, Leiny Garcia University of California, Irvine, Yvonne Kao WestEd, Diana Franklin University of Chicago, Susan Krause University of Chicago, Mark Warschauer University of California, Irvine
DOI Pre-print
11:00
30m
Paper
Using Adaptive Parsons Problems to Scaffold Write-Code Problems
Research Papers
Xinying Hou University of Michigan, Barbara Ericson University of Michigan, Xu Wang University of Michigan
DOI
11:30
30m
Paper
Automatic Generation of Programming Exercises and Code Explanations Using Large Language ModelsBest Paper
Research Papers
Sami Sarsa Aalto University, Paul Denny The University of Auckland, Arto Hellas Aalto University, Juho Leinonen Aalto University
DOI Pre-print