1. Home
  2. Vol 10, No 2 (2026)
  3. MUAMBA

Deep Q-Network-Based Adaptive Edge Intelligence for Energy-Efficient Air Quality Monitoring in IoT Networks.

Donatien KADIMA MUAMBA(1),


(1) Faculté des Sciences Informatiques/ Université Révérend KIM Ecole de Télécommunication et Télédétection Spatiale/ Université Pédagogique Nationale
Corresponding Author

Abstract


Summary The fast deployment of Internet of Things (IoT) networks has made ubiquitous environmental monitoring possible. However, the conventional air quality monitoring systems usually depend on a fixed sensing period, which leads to redundant energy consumption, large communication overhead and poor adaptability to dynamic ambient conditions. In this paper, we propose an adaptive edge intelligence framework based on Deep Q-Network (DQN) for energy-efficient air quality monitoring in the IoT networks. The proposed system integrates low-cost environmental sensors with an ESP32-based edge node for collecting important environmental parameters like particulate matter (PM2.5 and PM10), gaseous pollutants (CO₂, NOx, SOx, and NHx), temperature, humidity and traffic-related information. A DQN agent automatically chooses sensing and communication actions based on the sensed environmental state. We design a customized reward function that jointly takes into account the air quality conditions, sensing energy consumption, communication cost, and adaptive monitoring requirements, and the state representation we propose is composed of nine environmental and contextual variables. The experimental evaluation was performed with real measurements collected by the developed IoT platform and stored in a Supabase cloud database. The DQN agent was trained on 103 valid observations of the environment for 200 episodes and then tested on 114 observations. The total reward during training increased from −43.62 in the first episode to about 270 reward units after convergence. In the independent evaluation, the learned policy achieved a total reward of 316.092, an average reward of 2.773 per decision step and an average energy cost of 0.20 per step. Given relatively stable environmental conditions during the evaluation period, the agent chose the SLOW_SAMPLE strategy in all the decisions, thus avoiding unnecessary high-frequency sensing operations. These findings show the feasibility of integrating deep reinforcement learning with IoT edge intelligence for adaptive and energy-aware environment monitoring. The proposed framework provides a scalable foundation for self-optimizing IoT sensing systems that can adapt their operational behavior dynamically to environmental conditions

Keywords


Internet of Things (IoT); Edge Intelligence; Deep Q-Network (DQN); Reinforcement Learning; Air Quality Monitoring; Energy Efficiency; Adaptive Sampling; Smart Environment.

References


V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature, vol. 518, pp. 529–533, 2015, doi: 10.1038/nature14236.

V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, and M. Riedmiller, “Playing Atari with deep reinforcement learning,” arXiv preprint arXiv:1312.5602, 2013.

H. van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with Double Q-learning,” in Proc. AAAI Conf. Artificial Intelligence, vol. 30, no. 1, pp. 2094–2100, 2016, doi: 10.1609/aaai.v30i1.10295.

T. Schaul, J. Quan, I. Antonoglou, and D. Silver, “Prioritized experience replay,” in Proc. Int. Conf. Learning Representations (ICLR), 2016.

Z. Wang, T. Schaul, M. Hessel, H. van Hasselt, M. Lanctot, and N. de Freitas, “Dueling network architectures for deep reinforcement learning,” in Proc. Int. Conf. Machine Learning (ICML), 2016.

R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, 2nd ed. Cambridge, MA, USA: MIT Press, 2018.

J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of Things (IoT): A vision, architectural elements, and future directions,” Future Generation Computer Systems, vol. 29, no. 7, pp. 1645–1660, 2013.

W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE Internet of Things Journal, vol. 3, no. 5, pp. 637–646, 2016, doi: 10.1109/JIOT.2016.2579198.

Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” Proceedings of the IEEE, vol. 107, pp. 1738–1762, 2019, doi: 10.1109/JPROC.2019.2918951.

J. Lin, W. Yu, N. Zhang, X. Yang, H. Zhang, and W. Zhao, “A survey on Internet of Things: Architecture, enabling technologies, security and privacy, and applications,” IEEE Internet of Things Journal, vol. 4, no. 5, pp. 1125–1142, 2017.

C. Perera, C. H. Liu, and S. Jayawardena, “Context-aware computing in the Internet of Things: A survey on Internet of Things from industrial market perspective,” IEEE Communications Surveys & Tutorials, vol. 16, no. 1, pp. 414–454, 2014.

R. Li, Z. Zhao, Q. Sun, C.-L. I, C. Yang, X. Chen, M. Zhao, and H. Zhang, “Deep reinforcement learning for resource management in network slicing,” IEEE Access, vol. 6, pp. 74429–74441, 2018, doi: 10.1109/ACCESS.2018.2881964.

S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “A survey on mobile edge networks: Convergence of computing, caching and communications,” IEEE Access, vol. 5, pp. 6757–6779, 2017.

Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE Communications Surveys & Tutorials, vol. 19, no. 4, pp. 2322–2358, 2017.

World Health Organization, WHO Global Air Quality Guidelines: Particulate Matter (PM2.5 and PM10), Ozone, Nitrogen Dioxide, Sulfur Dioxide and Carbon Monoxide. Geneva, Switzerland: World Health Organization, 2021.

A. Kumar, I. P. Singh, and S. P. Singh, “IoT-based air quality monitoring and forecasting systems: A review,” IEEE Sensors Journal, [publication details to be verified according to the final selected source].

A. Al-Fuqaha, M. Guizani, M. Mohammadi, M. Aledhari, and M. Ayyash, “Internet of Things: A survey on enabling technologies, protocols, and applications,” IEEE Communications Surveys & Tutorials, vol. 17, no. 4, pp. 2347–2376, 2015.

F. Samie, L. Bauer, and J. Henkel, “From cloud down to things: An overview of machine learning in Internet of Things,” IEEE Internet of Things Journal, vol. 6, no. 3, pp. 4921–4934, 2019.

M. Chen, Y. Miao, X. Jia, and K. Hwang, “Intelligent resource management for Internet of Things based on deep reinforcement learning,” IEEE Internet of Things Journal, [publication details to be verified according to the final selected source].

W. Zhang, Y. Wen, K. Guan, D. Kilper, and H. Luo, “Energy-optimal mobile cloud computing under stochastic wireless channel,” IEEE Transactions on Wireless Communications, vol. 12, no. 9, pp. 4569–4581, 2013.


Article Metrics

Abstract View : 0 times


DOI: 10.56327/ijiscs.v10i2.1905

Refbacks

  • There are currently no refbacks.