1.
Li, Haiyuan; Li, Peizheng; Assis, Karcuis; Ullauri, Juan Marcelo Parra; Aijaz, Adnan; Yan, Shuangyi; Simeonidou, Dimitra
NetMind+: Adaptive Baseband Function Placement with GCN Encoding and Incremental Maze-solving DRL for Dynamic and Heterogeneous RANs Journal Article
In: IEEE Transactions on Network and Service Management, vol. 22, no. 4, pp. 3419–3432, 2025, ISSN: 1932-4537.
Abstract | Links | BibTeX | Tags: Advanced RAN, deep reinforcement learning, graph neural network, Incremental learning, MEC, Topology variation
@article{li_netmind_2025,
title = {NetMind+: Adaptive Baseband Function Placement with GCN Encoding and Incremental Maze-solving DRL for Dynamic and Heterogeneous RANs},
author = {Haiyuan Li and Peizheng Li and Karcuis Assis and Juan Marcelo Parra Ullauri and Adnan Aijaz and Shuangyi Yan and Dimitra Simeonidou},
doi = {10.1109/TNSM.2025.3570490},
issn = {1932-4537},
year = {2025},
date = {2025-08-01},
journal = {IEEE Transactions on Network and Service Management},
volume = {22},
number = {4},
pages = {3419–3432},
abstract = {The disaggregated architecture of advanced Radio Access Networks (RANs) with diverse X-haul latencies, in conjunction with resource-limited multi-access edge computing networks, presents significant challenges in designing a general model in placing baseband and user plane functions to accommodate versatile 5G services. This paper proposes a novel approach, NetMind+, which leverages Deep Reinforcement Learning (DRL) to determine the function placement strategies in diverse and evolving RAN topologies, aiming at minimizing power consumption. NetMind+ resolves the problem with a maze-solving strategy, enabling a Markov Decision Process with standardized action space scales across different networks. Additionally, a Graph Convolutional Network (GCN) based encoding and an incremental learning mechanism are introduced, allowing features from different and dynamic networks to be aggregated into a single DRL agent. This facilitates the generalization capability of DRL and minimizes the negative retraining impact. In an example with three sub-networks, NetMind+ demonstrates a substantial 32.76% improvement in power savings and a 41.67% increase in service stability compared to benchmarks from the existing literature. Compared to traditional methods necessitating a dedicated DRL agent for each network, NetMind+ attains comparable performance with 70% of the training cost savings. Furthermore, it demonstrates robust adaptability during network variations, accelerating training speed by 50%.},
keywords = {Advanced RAN, deep reinforcement learning, graph neural network, Incremental learning, MEC, Topology variation},
pubstate = {published},
tppubtype = {article}
}
The disaggregated architecture of advanced Radio Access Networks (RANs) with diverse X-haul latencies, in conjunction with resource-limited multi-access edge computing networks, presents significant challenges in designing a general model in placing baseband and user plane functions to accommodate versatile 5G services. This paper proposes a novel approach, NetMind+, which leverages Deep Reinforcement Learning (DRL) to determine the function placement strategies in diverse and evolving RAN topologies, aiming at minimizing power consumption. NetMind+ resolves the problem with a maze-solving strategy, enabling a Markov Decision Process with standardized action space scales across different networks. Additionally, a Graph Convolutional Network (GCN) based encoding and an incremental learning mechanism are introduced, allowing features from different and dynamic networks to be aggregated into a single DRL agent. This facilitates the generalization capability of DRL and minimizes the negative retraining impact. In an example with three sub-networks, NetMind+ demonstrates a substantial 32.76% improvement in power savings and a 41.67% increase in service stability compared to benchmarks from the existing literature. Compared to traditional methods necessitating a dedicated DRL agent for each network, NetMind+ attains comparable performance with 70% of the training cost savings. Furthermore, it demonstrates robust adaptability during network variations, accelerating training speed by 50%.
2.
Li, Haiyuan; Madhukumar, Hari; Li, Peizheng; Liu, Yuelin; Teng, Yiran; Wu, Yulei; Wang, Ning; Yan, Shuangyi; Simeonidou, Dimitra
Toward Practical Operation of Deep Reinforcement Learning Agents in Real-World Network Management at Open RAN Edges Journal Article
In: IEEE Communications Magazine, 2025, ISSN: 0163-6804.
Abstract | Links | BibTeX | Tags: DRL, MEC, Network management and orchestration, O-RAN, Practical deployment
@article{li_toward_2025,
title = {Toward Practical Operation of Deep Reinforcement Learning Agents in Real-World Network Management at Open RAN Edges},
author = {Haiyuan Li and Hari Madhukumar and Peizheng Li and Yuelin Liu and Yiran Teng and Yulei Wu and Ning Wang and Shuangyi Yan and Dimitra Simeonidou},
doi = {10.1109/MCOM.001.2500207},
issn = {0163-6804},
year = {2025},
date = {2025-07-01},
journal = {IEEE Communications Magazine},
abstract = {Deep Reinforcement Learning (DRL) has emerged as a powerful solution for meeting the growing demands for connectivity, reliability, low latency and operational efficiency in advanced networks. However, most research has focused on theoretical analysis and simulations, with limited investigation into real-world deployment. To bridge the gap and support practical DRL deployment for network management, we first present an orchestration framework that integrates ETSI Multi-access Edge Computing (MEC) with Open RAN, enabling seamless adoption of DRL-based strategies across different time scales while enhancing agent lifecycle management. We then identify three critical challenges hindering DRL's real-world deployment, including asynchronous requests from unpredictable or bursty traffic, adaptability and generalization across heterogeneous topologies and evolving service demands, and prolonged convergence and service interruptions due to exploration in live operational environments. To address these challenges, we propose a three-fold solution strategy: advanced time-series integration for handling asynchronized traffic, flexible architecture design such as multi-agent DRL and incremental learning to support heterogeneous scenarios, and simulation-driven deployment with transfer learning to reduce convergence time and service disruptions. Lastly, the feasibility of the MEC-O-RAN architecture is validated on an urban-wide testing infrastructure, and two real-world use cases are presented, showcasing the three identified challenges and demonstrating the effectiveness of the proposed solutions.},
keywords = {DRL, MEC, Network management and orchestration, O-RAN, Practical deployment},
pubstate = {published},
tppubtype = {article}
}
Deep Reinforcement Learning (DRL) has emerged as a powerful solution for meeting the growing demands for connectivity, reliability, low latency and operational efficiency in advanced networks. However, most research has focused on theoretical analysis and simulations, with limited investigation into real-world deployment. To bridge the gap and support practical DRL deployment for network management, we first present an orchestration framework that integrates ETSI Multi-access Edge Computing (MEC) with Open RAN, enabling seamless adoption of DRL-based strategies across different time scales while enhancing agent lifecycle management. We then identify three critical challenges hindering DRL's real-world deployment, including asynchronous requests from unpredictable or bursty traffic, adaptability and generalization across heterogeneous topologies and evolving service demands, and prolonged convergence and service interruptions due to exploration in live operational environments. To address these challenges, we propose a three-fold solution strategy: advanced time-series integration for handling asynchronized traffic, flexible architecture design such as multi-agent DRL and incremental learning to support heterogeneous scenarios, and simulation-driven deployment with transfer learning to reduce convergence time and service disruptions. Lastly, the feasibility of the MEC-O-RAN architecture is validated on an urban-wide testing infrastructure, and two real-world use cases are presented, showcasing the three identified challenges and demonstrating the effectiveness of the proposed solutions.