Scalex: Scalability Exploration of Multi-Agent Reinforcement Learning Agents in Grid-Interactive Efficient Buildings

Abstract

Transitioning to renewable energy and decarbonization presents challenges for grid-interactive efficient building (GEB) communities. Conventional control systems struggle to maximize intermittent renewable energy, but advanced control architecture and utilization of renewable sources with energy storage can overcome this limitation and optimize energy flexibility. Reinforcement learning (RL) offers potential solutions, but its scalability and computational demands in large-scale settings remain unclear. This paper examines the scalability of Soft-Actor Critic (SAC) in multi-agent systems, comparing decentralized-independent SACs and centralized SACs using CityLearn, an OpenAI Gym environment. We consider neighborhoods consisting of 2 to 64 single-family residential buildings, each equipped with cooling and heating storage devices, domestic hot water storage devices, electrical storage devices, and solar PV systems. Our findings suggest that independent controllers outperform the centralized controller with increasing number of buildings. We also show that the performance on the building level can differ from the aggregated performance.