Investigating the effectiveness of gestural interface features in a vocabulary learning game for children

Abstract

The purpose of this research is to examine the effect of using gestural interfaces in vocabulary learning. This thesis explores the use of embodied learning (EL) in vocabulary acquisition; in particular, the use of a touchless gestural interface to support embodied language learning. Prior work on supporting vocabulary acquisition using EL has suggested that employing congruent/semantic gestures related to the meaning of a word (e.g., representing the word “book” by opening and closing the hands, palms facing up) while learning helps young learners to recall new words later. Touchless gestural interfaces (which allow users to interact with digital content on a large display by using body movements/gestures without holding or wearing any devices) have good potential to support vocabulary acquisition using embodied learning in classrooms. However, there has been very little work investigating students’ interactions and experiences with such interfaces in language learning contexts, and no previous work has compared different types of gestures when learning words using touchless gestural interfaces. In addition, to encourage children to perform gestures when using gestural interfaces, the majority of existing systems use automatic detection techniques, which can cause frustration when detection fails and can limit the range of gestures that can be used. This thesis aims to address these gaps by conducting three empirical studies exploring students’ interactions and experiences when using touchless gestural interfaces in vocabulary acquisition games. This thesis also investigates the experience and efficacy of different gesture types and the use of a 3D mirroring avatar to incorporate gestures into a learning game, which encourages gestures without the need for automatic detection.