Automation of vehicular systems using deep reinforcement learning and mean-field models: Application to heavy duty trucks

Abstract

The transportation sector consumes about a third of all energy consumed in the world, about a third of which is consumed by trucks. Future transportation systems must address this energy challenge, in addition to the other inefficiencies related to time, money, and lives lost while the system is operating. Vehicle automation is one of the promising opportunities underway. For instance, cooperative adaptive cruise control, an extension of the more popular cruise control and adaptive cruise control systems, promises to reduce fuel consumption by up to 15% for participating trucks, reduce emissions, increase road capacity at high technology penetration rates, and contribute to road safety.

Heavy duty trucks are complex vehicles that are designed and built for specific mission requirements. Any of these trucks could be equipped from a wide selection of vehicle components with a significantly wide spectrum of operating dynamics and performances. Driving a heavy duty truck is an equally complex task. Human drivers must be well educated and trained about the specific truck they are about to drive and operate. They must optimize in real-time for factors such as truck dynamics and driving performance; road, truck, and payload safety; truck operation economics; truck driving law constraints; mission constraints; in addition to background traffic on the road.

Automation of heavy duty truck operation tasks require equally advanced engineering tools. For instance, high precision modeling and control have historically required a detailed study of each subject truck. This thesis presents a process using deep learning and deep reinforcement learning for microscopic longitudinal modeling and control of such trucks that is agnostic to their internal mechanics. The process is demonstrated and evaluated for several truck mechanical configurations using high fidelity simulation and in the field. Cruise control of single truck operations has been considered, in addition to cooperative adaptive cruise control for multi-truck coordination.

Long haul heavy duty trucks often drive within shared road infrastructure with background traffic. To account for this traffic on the road, we consider multi-scale partial differential equation mean-field models. With this approach, each truck is modeled microscopically while background traffic is modeled mesoscopically. A nondissipative numerical solver is developed and evaluated for computational study of these models. The solver maintains structure and resolution at a wide range of discretization resolutions suitable for development of optimal control laws.

This thesis investigates computational methods for the automation of heavy duty trucks. While vehicle driving automation is already underway, more investigation is still required to bring about full autonomy. The future of the transportation system and trucking could benefit from further study and development of the sciences and engineering of autonomy with consideration to the complex interplay between the vehicle as an agent, the transportation system as an operations context, the logistics system as a mission context, and the human beneficiary.