Human Activity Monitoring for Telemedicine Using an Intelligent Millimeter-Wave System

Abstract

The growing aging population requires innovative solutions in the healthcare industry. Telemedicine is one such innovation that can improve healthcare access and delivery to diverse and aging populations. It uses various sensors to facilitate remote monitoring of physiological measures of people, such as heart rate, oxygen saturation, blood glucose, and blood pressure. Similarly, it is capable of monitoring critical events, such as falls. The key challenges in telemonitoring are ensuring accurate remote monitoring of physical activity or falls by preserving privacy and avoiding excessive reliance on expensive and/or obtrusive devices.

Our approach initially addressed the need for secure, portable, and low-cost solutions specifically for fall detection. Our proposed system integrates a low-power millimeter-wave (mmWave) sensor with a NVIDIA Jetson Nano system and uses machine learning to accurately and remotely detect falls. Our initial work focused on processing the mmWave sensor's output by using neural network models, mainly employing Doppler signatures and a Long Short-Term Memory (LSTM) architecture. The proposed system achieved 79% accuracy in detecting three classes of human activities. In addition to reasonable accuracy, the system protected privacy by not recording camera images, ensuring real-time fall detection and Human Activity Recognition (HAR) for both single and multiple individuals at the same time.

Building on this foundation, we developed an advanced system to enhance accuracy and robustness in continuous monitoring of human activities. This enhanced system also utilized a mmWave radar sensor (IWR6843ISK-ODS) connected to a NVIDIA Jetson Nano board, and focused on improving the accuracy and robustness of the monitoring process. This integration facilitated effective data processing and inference at the edge, making it suitable for telemedicine systems in both residential and institutional settings. By developing a PointNet neural network for real-time human activity monitoring, we achieved an inference accuracy of 99.5% when recognizing five types of activities: standing, walking, sitting, lying, and falling. Furthermore, the proposed system provided activity data reports, tracking maps, and fall alerts and significantly enhanced telemedicine applcations by enabling more timely and targeted interventions based on objective data.

The final proposed system facilitates the ability to detect falls and monitor physical activity at both home and institutional settings, demonstrating the potential of Artificial Intelligence (AI) algorithms and mmWave sensors for HAR. In conclusion, our system enhances therapeutic adherence and optimizes healthcare resources by enabling patients to receive physical therapy services remotely. Furthermore, it could reduce the need for hospital visits and improve in-home nursing care, thus saving time and money and improving patient outcomes.