A SMARTPHONE-BASED MICRO-NAVIGATION SYSTEM FOR BLIND AND VISUALLY IMPAIRED PEOPLE

Abstract

The growing number of blind and visually impaired people (BVI), especially in low-income countries, has made it imperative to develop affordable electronic travel aid (ETA) systems that utilize smartphones' sensors and applications as all-in-one devices. These systems have the potential to be cost-effective and reasonable for BVI individuals.

This thesis proposes a single-camera smartphone-based pipeline for detecting obstacles in unfamiliar indoor and outdoor environments. The pipeline utilizes a monocular vision system to generate dense disparity maps. These maps are then used to detect obstacles and determine their distance from the phone. By utilizing a smartphone's Inertial Measurement Unit (IMU) to recognize gait, a novel approach is introduced to extract frames based on gait by detecting steps and turns. This approach ensures maximum lateral displacements during walking, enhancing depth perception. Furthermore, a quality measurement is employed to extract frames with minimal blur. This method reduces the number of frames required for processing, thereby reducing system loads and preserving smartphone battery life.

One of the challenges in estimating disparity maps is accurate disparity range estimation and rectification. Consequently, this thesis investigates various arrangements of feature detectors and descriptors to estimate a robust fundamental matrix for rectification purposes. Empirical and comprehensive evaluations using six datasets captured in indoor and outdoor environments demonstrate that the AKAZE-RT-LMEDS combination delivers satisfactory performance compared to other arrangements evaluated on the same datasets. Additionally, a novel approach is proposed to estimate disparity ranges in both vertical and horizontal directions, improving system performance. The results obtained show that the proposed approach is fast and outperforms other existing approaches in the literature.

However, the estimated fundamental matrix is prone to defects, resulting in imperfect rectified images. This motivates the proposal of a new approach to address these imperfections. This thesis presents three novel approaches called 3W-stereo matching, A3W-stereo matching, and 3WCTF-stereo matching, which aim to obtain a dense disparity map from imperfect rectified images. All three approaches extend traditional stereo matching by searching for matches in the vertical direction as well. The proposed approaches demonstrate improvements in estimated disparity, with 3WCTF providing the best performance among the three. Additionally, two modified Star Census Transform (STAR-CT) pixel descriptors for cost functions are introduced to estimate the disparity map. The first descriptor, called AD-STAR-CT, improves upon the performance of STARCT. However, the second descriptor, AD-STAR-CT-MDT, shows a degradation in performance.

The final proposed algorithm takes disparity maps with radial effects as input to estimate the ground plane. It clusters pixels into obstacles and free spaces and estimates the distance to obstacles. Unlike previous approaches that use disparity maps from stereo vision systems, the proposed modified algorithm generalizes the segmentation to handle disparity maps from both monocular and stereo vision systems. In the case of monocular vision, where flat ground is interpreted as curves in the v-disparity map, the algorithm can still accurately detect obstacles. This algorithm is capable of detecting all types of obstacles, including discontinuities in the ground plane, and estimating their distances without the need for reprojection.