Robust Deep Reinforcement Learning Using Formal Verification

We propose a method to enhance the robustness and efficiency of deep reinforcement learning (DRL) by integrating formal verification techniques into the training loop. Our approach uses counterexamples generated by verification tools as corrective feedback to guide policy adjustments, enabling the agent to avoid unsafe actions and learn faster. Inspired by imitation learning, the verification tool acts as an expert that continuously refines the neural network when the agent’s policy fails. Experiments in challenging environments such as Frozen Lake and Sokoban demonstrate that our method yields substantial improvements in success rates and reduces the number of training episodes by up to 70%, all while significantly enhancing policy safety. We release the code and full reproducibility instructions at https://github.com/AvrahamRaviv/Robust-DRL-FV.