Path following and obstacle avoidance of tracked vehicle via deep reinforcement learning with model predictive control as reference

Ting-Chien Chen; Yu-Cheng Sung; Chai-Wei Hsu; Dun-Ren Liu; Shean-Jen Chen

doi:10.1117/12.2673641

9 August 2023 Path following and obstacle avoidance of tracked vehicle via deep reinforcement learning with model predictive control as reference

Ting-Chien Chen, Yu-Cheng Sung, Chai-Wei Hsu, Dun-Ren Liu, Shean-Jen Chen

Author Affiliations +

Proceedings Volume 12621, Multimodal Sensing and Artificial Intelligence: Technologies and Applications III; 126210C (2023) https://doi.org/10.1117/12.2673641
Event: SPIE Optical Metrology, 2023, Munich, Germany

Abstract

Deep reinforcement learning (DRL) is based on rigorous mathematical foundations and adjusts network parameters through interactions with the environment. The stability problem of maintaining a vehicle on a continuous path can be achieved by soft actor-critic (SAC). Furthermore, a model predictive control (MPC) with prediction and control horizons under multivariable constraints can precisely follow the path, but the disadvantage is its large computation. In this paper, a DRL control scheme with MPC is proposed to precisely and effectively implement the path following and obstacle avoidance of tracked vehicle. The DRL controller performs the effective obstacle avoidance and is also in accordance with MPC to precisely follow planning paths. To make the training more realistic, a data-driven state-space dynamic model of the tracked vehicle is first estimated via N4SID system identification algorithm. During the DRL training, the MPC output is used as the reward input of the DRL to learn the MPC characteristics and an additional reward function is designed specifically for obstacle avoidance. The parameters of the DRL agent are adjusted based on the environment input and the MPC output. After the training, the MPC can be skipped since it is used as a part of the reward function, and the DRL has learned to imitate the MPC while achieves obstacle avoidance. The simulation and experimental results show that the overall controller has high stability, accuracy, and efficiency.

Conference Presentation

Citation Download Citation

Ting-Chien Chen, Yu-Cheng Sung, Chai-Wei Hsu, Dun-Ren Liu, and Shean-Jen Chen "Path following and obstacle avoidance of tracked vehicle via deep reinforcement learning with model predictive control as reference", Proc. SPIE 12621, Multimodal Sensing and Artificial Intelligence: Technologies and Applications III, 126210C (9 August 2023); https://doi.org/10.1117/12.2673641

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available