Expert-generated and Auto-generated Socratic Tutoring Systems for Code Comprehension

Abstract

Programming skills are an increasingly vital part of many disciplines but can be challenging to teach and learn due to the inherent complexity of programming and the extensive practice needed to master such skills. Introductory programming courses are frequently considered difficult, frustrating, and a major stumbling block for students, with drop-out and failure rates often reaching 30-40%. To overcome these challenges, students could benefit from extensive individual support such as tutoring, but there are simply not enough qualified tutors available to meet rising demands. A potential solution is the development of intelligent tutoring systems (ITSs), which offer individualized, one-on-one instruction. Such systems can offer the much needed individual support to make computer programming instruction more effective and scalable and at the same time reduce existing teachers’ workloads. Such ITSs to help students in different phases of programming were developed as early as 1975. For instance, the xTEx-Sys was created to help students with the planning and design phases of writing a computer program. In addition, some ITSs have targeted programming concepts, such as OOPS and ProgTool, which focus solely on the class concept, while ProPL targets conditions and loops. However, authoring content for such systems requires cognitive science and programming expertise and is tedious, error-prone, and time-consuming. This dissertation demonstrates how conversational ITSs that rely on natural language technology and the Socratic method of teaching can improve a novice’s understanding of programming concepts and in particular the scaffolding of code comprehension processes. Furthermore, the work presented in this dissertation provides a novel method to automatically author a Socratic dialogue-based ITS. Indeed, two major outcomes of this work are a Socratic dialogue-based ITS called Socratic Tutor and an automated dialogue authoring tool called Socratic Author, which generates full Socratic dialogue from Java source code. The key objectives of this dissertation were, first, to determine whether the Socratic method would be effective at eliciting learners to engage in self-explanations with the help of the Socratic Tutor ITS and, second, assess the quality of Socratic Author’s auto-generated tutorial dialogue. Thus, the work presented here sought to answer two main research questions: (1) can a Socratic ITS lead to improved code comprehension? and (2) to what extent can Socratic dialogue be generated automatically? In sum, this research helps establish a relationship between code comprehension and, more broadly, the use of the Socratic method in learning computer programming. Furthermore, the work introduces a novel approach for generating Socratic dialogue from source code with examples for the Java programming language. The auto-authoring tool could help teachers and ITS developers create tutorial dialogues automatically from Java code without requiring non–domain knowledge. To the best of our knowledge, no such auto-generation of tutorial dialogues from source code has been done before and thus constituting a premiere.