Predictive Maintenance Model for IIoT-Based Manufacturing: A Transferable Deep Reinforcement Learning Approach

Kevin Shen Hoong Ong, Wenbo Wang, Nguyen Quang Hieu, Dusit Niyato, Thomas Friedrichs

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

The Industrial Internet of Things (IIoT) is crucial for accurately assessing the state of complex equipment in order to perform predictive maintenance (PdM) successfully. However, existing IIoT-based PdM frameworks do not consider the influence of various practical yet complex system factors, such as the real-time production states, machine health, and maintenance manpower resources. For this reason, we propose a generic PdM optimization framework to assist maintenance teams in prioritizing and resolving maintenance task conflicts under real-world manufacturing conditions. Specifically, the PdM framework aims to jointly optimize the edge-based machine network uptime and the allocation of manpower resources in a stochastic IIoT-enabled manufacturing environment using the model-free deep reinforcement learning (DRL) methods. Since DRL requires a significant amount of training data, we propose and demonstrate the use of the transfer learning (TL) method to assist DRL in learning more efficiently by incorporating expert demonstrations, termed TL with demonstrations (TLDs). TLD reduces training wall time by 58% compared to baseline methods, and we conduct numerous experiments to illustrate the performance, robustness, and scalability of TLD. Finally, we discuss the general benefits and limitations of the proposed TL method, which are not well addressed in the existing literature but could be beneficial to both researchers and industry practitioners.

Original languageEnglish
Pages (from-to)15725-15741
Number of pages17
JournalIEEE Internet of Things Journal
Volume9
Issue number17
DOIs
StatePublished - 1 Sep 2022

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Decision support
  • Industrial Internet of Things (IIoT)
  • deep reinforcement learning (DRL)
  • predictive maintenance (PdM)
  • resource management
  • transfer learning (TL)

Fingerprint

Dive into the research topics of 'Predictive Maintenance Model for IIoT-Based Manufacturing: A Transferable Deep Reinforcement Learning Approach'. Together they form a unique fingerprint.

Cite this