Learning to Design: Reinforcement Learning Approaches to Layout Design Optimization with Simulated Crowds

Abstract

Layout planning prior to construction is a fundamental task in computer-aided design (CAD). Layout generation has been extensively studied within the computer graphics community through a range of methods and applications. A crucial yet neglected direction of research is to enable different crowd behaviour influence the layouts during the generation process. Integrating crowd behaviour into design can enhance safety management, support functional objectives, and accommodate the various needs multiple stakeholders. However, progress in this direction is constrained by the lack of suitable datasets. This limitation motivates the adoption of methods such as reinforcement learning, which do not rely primarily on training data. Reinforcement learning learn policies for optimal action sequences that modify the user space states. It offers transparency during training, unlike black-box models, allowing each step of the process to be evaluated and adjusted. This transparency facilitates the integration of multiple objectives, such as enabling the inclusion of real-time crowd feedback directly within the learning loop. However, reinforcement learning heavily relies on a well-defined reward function to guide learning toward the desired objectives. In practice, such rewards formulations are often incomplete or completely unavailable. Especially in complex or poorly understood environments. This challenge motivates research into approaches that operate without explicit reward functions. Instead, learning can be guided by implicit feedback or supported by pre-trained models. This thesis investigates both traditional reward-based reinforcement learning and implicit reward learning with the support of diffusion model. The research identifies key limitations and explores practical challenges, with particular attention to the role of crowd behaviour in shaping the designing process. The primary contribution consists of two approaches to design layouts. Both employ reinforcement learning to address data scarcity and include crowd simulation to model crowd behaviour in the environment. The first approach uses explicit reward functions. The second incorporates a diffusion model to overcome the need for an explicit reward specification that addresses the diversity of design principles. This research demonstrates the potential of combining reinforcement learning, diffusion model, and crowd simulation to address the challenges of layout planning for multiple objectives.