Understanding the Effects of Vision Impairment on Visual Search Through Computational Models

Abstract

This PhD thesis explores the application of computational models to help better understand the effects of vision loss on visual search behaviours. It analyses how people adapt to cognitive limitations when interacting with a visual search task. Visual search is a cognitive function that plays a crucial role in everyday activities, and gaining insights into how it differs amongst individuals with vision impairment holds significant practical implications for designing more accessible user interfaces. The research is exploratory and interpretative. It examines the implementation of a cognitive information processing system, which serves as the thinking brain to a computational model (i.e. the mechanism). Specifically, it explores how the mechanism affects visual search strategies in vision impairment. Researchers have used computational-rational agents to analyse and predict eye movement strategies. However, most studies have only focused on the strategies of normal vision without considering vision impairment. Therefore, in this research, we are taking a new approach by attempting to construct a model of eye movements in individuals with different types of vision impairment. To achieve optimal adaptation strategies in the model, we formulate different visual search tasks as a Partially Observable Markov Decision Process (POMDP). This approach would allow the implementation of the constraints imposed by the environment and the human visual system, including the limitations resulting from vision impairment. Solving real-life problems as POMDP is complex and requires significant computational resources. Therefore, we use different Reinforcement Learning (RL) techniques to achieve optimal strategies that we can compare against human data. The thesis considers three types of vision impairments (Glaucoma, Cataracts, and Age-related Macular Degeneration (AMD)). We use the Q-learning method to find the optimal policy for the Glaucoma task. Whereas for the other tasks, we use Deep Reinforcement Learning (DRL), specifically Proximal Policy Optimisation (PPO). The model finds optimal strategies in each experiment through adaptation to an impaired visual field. The results show that the theoretical approach to optimal rationality can give insight into how people with various types of vision impairment would interact with the screen. Overall, the model achieved the following: (1) adapted strategies to peripheral vision and environment constraints, (2) it showed a decline in performance when given a crowded setting, and (3) it predicted the accuracy of people with Cataracts in two different tasks. The research findings suggest that people with vision impairments adapt to their types of vision loss and the nature of their visual field defect. The findings have significant implications for Human-Computer Interaction (HCI). They can provide insights into the interactions of people with a specific type of vision impairment.