Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization

Pengqi Sun, Chunxi Yang, Xiaojie Zhou, Wenbo Wang

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

To solve the problems of path planning and dynamic obstacle avoidance for an unmanned surface vehicle (USV) in a locally observable non-dynamic ocean environment, a visual perception and decision-making method based on deep reinforcement learning is proposed. This method replaces the full connection layer in the Proximal Policy Optimization (PPO) neural network structure with a convolutional neural network (CNN). In this way, the degree of memorization and forgetting of sample information is controlled. Moreover, this method accumulates reward models faster by preferentially learning samples with high reward values. From the USV-centered radar perception input of the local environment, the output of the action is realized through an end-to-end learning model, and the environment perception and decision are formed as a closed loop. Thus, the proposed algorithm has good adaptability in different marine environments. The simulation results show that, compared with the PPO algorithm, Soft Actor-Critic (SAC) algorithm, and Deep Q Network (DQN) algorithm, the proposed algorithm can accelerate the model convergence speed and improve the path planning performances in partly or fully unknown ocean fields.

Original languageEnglish
JournalSensors
Volume23
Issue number21
DOIs
StatePublished - 31 Oct 2023
Externally publishedYes

Keywords

  • USV
  • deep neural network
  • deep reinforcement learning
  • generalization
  • path planning
  • perception

Fingerprint

Dive into the research topics of 'Path Planning for Unmanned Surface Vehicles with Strong Generalization Ability Based on Improved Proximal Policy Optimization'. Together they form a unique fingerprint.

Cite this