Detecting Makeup Activities using Internet-of-Things

Abstract

This thesis focuses on identifying human activities for rendering make-up activities using sensors’ data and a supervised machine learning approaches. We considered five make-up activities in our work, such as, applying cream, lipsticks, blusher, eyeshadow, and mascara. We collected the data from ten participants using two smart-watch built-in sensors, accelerometer and gyroscope. We preprocessed the data and trained with different predictive machine learning models and we evaluated make-up activity prediction built on using Naïve Bayes, Simple Logistic, k-nearest neighbors’, and the random forest algorithms. We investigated the models' performance on three different datasets that differ by the environment they were collected in. The first dataset was collected from the participants using a controlled environment. In this staged setting, we provided the participants specific instructions on how to perform the five make-up activities. The second dataset was collected from the participants in an uncontrolled environment. We did not inform the participants with any prior instructions on how to perform the five activities and therefore, naturally they performed the make-up activities in their own way. Third, we synthetically generated a dataset by combining the existing datasets from the participants who were under both controlled and uncontrolled environments. Our results showed a 92.7 % accuracy for the controlled environment case given by the Gradient Boosting classifier and an 89.20 % accuracy for the uncontrolled environment case shown by the Random Forest classifier. Finally, Random Forest classifier registered the highest accuracy 92%, for the hybrid case where both the datasets from controlled and the uncontrolled environments were combined. We believe that this early work on recognizing and discovering a multitude of make-up activities has potential application in assessing and training the performance of various stakeholders in the future work of fashion industry.