Soft Actor-Critic for Autonomous Race Car Control

  This thesis implements a reinforcement learning approach for optimizing the trajectory of  a race car about a given track in real-time provided observations of the state of the car, in  addition to the state of the racetrack with respect to the car. A Soft Actor-Critic (SAC)  model operating in a continuous space implemented with Stable Baselines 3 was trained and  tested on a fork of BeamNG’s Gymnasium wrapper repository. The Soft Actor-Critic agent  was provided vehicle state information such as the velocity, angle relative to the centerline,  revolutions per minute (RPM), and simulated light detection and ranging (LiDAR) readings  of the edges of the racetrack to provide an observation closely approximating the current  state of the car. A custom reward function was generated prioritizing large track progress  rates with a penalty for excessive steering rates to minimize oscillatory movement of the car  keeping it minimal to avoid generating results which do not align with the main objective of  optimizing the lap times. A training pipeline was developed where the model was initially  trained on a small percentage of the racetrack, which was incrementally increased until  the model was being trained on 100% of the racetrack. The performance of the agent  was evaluated for the average lap completion percentage in addition to the average lap  time if applicable. The agent was also evaluated on different vehicle types to study the  transferability of the SAC model