Skip to main content
placeholder image

Toward Deep Q-Network-Based Resource Allocation in Industrial Internet of Things

Journal Article


Abstract


  • With the increasing adoption of Industrial Internet-of-Things (IIoT) devices, infrastructures, and supporting applications, it is critical to design schemes to effectively allocate resources (e.g., networking, computing, and energy) in IIoT systems, generally formalized as optimization problems. Nonetheless, because the system is highly complex, operation and networking graph-based environments are time varying, and required information may not be available, it is difficult to leverage traditional optimization techniques to solve the optimal resource allocation problem. In this article, we propose a deep Q-network (DQN)-based scheme to address both bandwidth utilization and energy efficiency in a networking graph-based IIoT system. In detail, we design a DQN model that consists of two deep neural networks (DNNs) and a Q-learning model. The DNN network abstracts the features from the highly dimensional inputs and obtains the approximate Q-function for the Q-learning model. Based on the Q-function, the Q-learning model can generate the Q-table and reward function. After the training process, the DQN model can select appropriate actions for the agents (i.e., robots in a smart warehouse in this study) to improve bandwidth utilization and energy efficiency. To evaluate our proposed scheme, we design a simulation environment to investigate a typical IIoT scenario: the actuation of robotics in a smart warehouse. We then implement the DQN model and conduct extensive experiments to validate the efficacy of our scheme. Our experimental results confirm that our scheme can improve both bandwidth utilization and energy efficiency, as compared to other representative schemes.

Publication Date


  • 2022

Citation


  • Liang, F., Yu, W., Liu, X., Griffith, D., & Golmie, N. (2022). Toward Deep Q-Network-Based Resource Allocation in Industrial Internet of Things. IEEE Internet of Things Journal, 9(12), 9138-9150. doi:10.1109/JIOT.2021.3093346

Scopus Eid


  • 2-s2.0-85112135564

Web Of Science Accession Number


Start Page


  • 9138

End Page


  • 9150

Volume


  • 9

Issue


  • 12

Abstract


  • With the increasing adoption of Industrial Internet-of-Things (IIoT) devices, infrastructures, and supporting applications, it is critical to design schemes to effectively allocate resources (e.g., networking, computing, and energy) in IIoT systems, generally formalized as optimization problems. Nonetheless, because the system is highly complex, operation and networking graph-based environments are time varying, and required information may not be available, it is difficult to leverage traditional optimization techniques to solve the optimal resource allocation problem. In this article, we propose a deep Q-network (DQN)-based scheme to address both bandwidth utilization and energy efficiency in a networking graph-based IIoT system. In detail, we design a DQN model that consists of two deep neural networks (DNNs) and a Q-learning model. The DNN network abstracts the features from the highly dimensional inputs and obtains the approximate Q-function for the Q-learning model. Based on the Q-function, the Q-learning model can generate the Q-table and reward function. After the training process, the DQN model can select appropriate actions for the agents (i.e., robots in a smart warehouse in this study) to improve bandwidth utilization and energy efficiency. To evaluate our proposed scheme, we design a simulation environment to investigate a typical IIoT scenario: the actuation of robotics in a smart warehouse. We then implement the DQN model and conduct extensive experiments to validate the efficacy of our scheme. Our experimental results confirm that our scheme can improve both bandwidth utilization and energy efficiency, as compared to other representative schemes.

Publication Date


  • 2022

Citation


  • Liang, F., Yu, W., Liu, X., Griffith, D., & Golmie, N. (2022). Toward Deep Q-Network-Based Resource Allocation in Industrial Internet of Things. IEEE Internet of Things Journal, 9(12), 9138-9150. doi:10.1109/JIOT.2021.3093346

Scopus Eid


  • 2-s2.0-85112135564

Web Of Science Accession Number


Start Page


  • 9138

End Page


  • 9150

Volume


  • 9

Issue


  • 12