Competence Assessment by Stimulus Matching (CASM): A Novel Approach to Language Assessment by Chunk Transcription

Abstract

This thesis develops and evaluates Competence Assessment by Stimulus Matching (CASM) as an innovative approach to assessing individual’s competence in English as a second language, integrating chunking theory from cognitive science with elements of human-computer interaction. The research addresses the need for more efficient and effective methods to assess language competence. It explores the potential for extracting chunk signals from microbehaviours observed during interactions in simple mouse-based tasks, as a reflection of language proficiency. The first step in the research was the development of a cognitive model utilizing the GOMS task analysis framework to inform the design of CASM. Following this, three empirical studies were conducted with eighty-nine participants who speak Arabic as their first language and English as their second, to test, refine and assess the effectiveness of the method. Chunk measures, such as pauses between clicks (pauses), the number of times a stimulus is viewed (view number), the duration of each view (view duration), and the duration of time spent clicking on answers between views (response duration) were collected and contrasted with independent measures of competence that included a language self-assessment, a vocabulary size test and an additional grammar test. The findings reveal a significant relationship between the dependant chunk measure and independent measures of language competence, demonstrating potential for using CASM for language assessment. This study highlights the value of exploring nontraditional ways of testing using technological advances to extract rich data. It also provides key considerations to assess competence using chunk signals across various domains.