Deciphering Hand Movement Patterns During Driving Using Smartwatch Signals Without Ground Truth

Abstract

We developed a method to identify atypical hand movements in driving, some of which are associated with detachment from the steering wheel and, thus, physical distraction. We performed our data analysis on NUBI { a naturalistic dataset collected from a week-long observation of n=57 Texas drivers. NUBI features data from over 900 trips with a total duration of over 300 hours. Due to a lack of visual ground truth, we employed unsupervised learning methods. Thanks to the GPS data to our avail, we used information about the type of road (highway or city street) and the type of segment (straight or turn) to narrow our search space. In more detail, we extracted features from the drivers' smartwatch motion signals using Temporal Convolutional Autoencoder (TCNAE). Then, we fed these encoded features into a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm. DBSCAN produced a main cluster and the remainder. The remainder is consistently associated with behaviors in turns and other atypical scenarios, such as queuing to pick up orders from fast-food dispensing windows. The characteristics of these atypical patterns are so distinct from the typical driving patterns (main cluster) that a random forest classi cation algorithm attained 99% area under the curve (AUC) performance in a ve-fold cross-validation test. Based on the kinematic constraints of the driver's hands, we developed a physics-based formula that associates elbow angles with gravitational acceleration values. We estimated the gravitational acceleration values that correspond to hand detachment from the steering wheel (i.e., extreme elbow angles). Applying these thresholds to the NUBI dataset, we found that such steering wheel detachment values arise just outside the dispensing windows of fast-food chains, where the drivers must pick up their orders. This nding validates our estimation method. In all, our approach not only nds atypical hand-motion patterns in driving but also pinpoints among these atypical patterns the patterns that involve hand detachment from the steering wheel. The latter are associated with physical distractions and crash risk under certain conditions. Notably, our approach achieves all these from smartwatch signals alone without any need to resort to visual ground truth from camera feeds. Given the ubiquity of smartwatches and the unavailability of cameras in car interiors, the practical implications of this development cannot be overestimated.