Designing a Sensor-Based Wearable Computing System for Custom Hand Gesture Recognition Using Machine Learning

Abstract

This thesis investigates how assistive technology can be made to facilitate communication for people that are unable to or have dif- ficulty communicating via vocal speech, and how this technology can be made more universal and compatible with the many di↵er- ent types of sign language that they use. Through this research, a fully customisable and stand-alone wearable device was developed, that employs machine learning techniques to recognise individual hand gestures and translate them into text, images and speech. The device can recognise and translate custom hand gestures by train- ing a personal classifier for each user, relying on a small training sample size, that works o✏ine on an embedded system or mobile device, with a classification accuracy rate of up to 99%. This was achieved through a series of iterative case studies, with user testing carried out by real users in their every day environments and in public spaces.