1.
Singh, Jitendra; Jagannatham, Aditya K.; Hanzo, Lajos
Geometric Mean Rate Maximization in RIS-Aided mmWave ISAC Systems Relying on a Non-Diagonal Phase Shift Matrix Journal Article
In: IEEE Open Journal of the Communications Society, vol. 6, pp. 4756–4771, 2025, ISSN: 2644-125X.
Abstract | Links | BibTeX | Tags: and geometric mean rate, Array signal processing, Base stations, Copper, Costs, Hardware, Integrated sensing and communication, millimeter wave, Millimeter wave communication, Optimization, Radio frequency, reconfigurable intelligent surface, Reconfigurable Intelligent Surfaces
@article{singh_geometric_2025,
title = {Geometric Mean Rate Maximization in RIS-Aided mmWave ISAC Systems Relying on a Non-Diagonal Phase Shift Matrix},
author = {Jitendra Singh and Aditya K. Jagannatham and Lajos Hanzo},
url = {https://ieeexplore.ieee.org/document/11012749/similar},
doi = {10.1109/OJCOMS.2025.3573196},
issn = {2644-125X},
year = {2025},
date = {2025-01-01},
urldate = {2025-10-08},
journal = {IEEE Open Journal of the Communications Society},
volume = {6},
pages = {4756–4771},
abstract = {The joint optimization of the hybrid transmit precoders (HTPCs) and reflective elements of a millimeter wave (mmWave) integrated sensing and communication (ISAC) system is considered. The system also incorporates a reconfigurable intelligent surface (RIS) relying on a non-diagonal RIS (NDRIS) phase shift matrix. Specifically, we consider a hybrid architecture at the ISAC base station (BS) that serves multiple downlink communication users (CUs) via the reflected links from the RIS, while concurrently detecting multiple radar targets (RTs). We formulate an optimization problem that aims for maximizing the geometric mean (GM) rate of the CUs, subject to the sensing requirement for each RT. Additional specifications related to the limited transmit power and unit modulus (UM) constraints for both the HTPCs and the reflective elements of the NDRIS phase shift matrix make the problem challenging. To solve this problem, we first transform the intractable GM rate expression to a tractable weighted sum rate objective and next split the transformed problem into sub-problems. Consequently, we propose an iterative alternating optimization approach that leverages the majorization-minimization (MM) framework and block coordinate descent (BCD) method to solve each sub-problem. Furthermore, to tackle the UM constraints in the sub-problem of the HTPC design, we propose a penalty-based Riemannian manifold optimization (PRMO) algorithm, which optimizes the HTPCs on the Riemannian manifold. Similarly, the phases of the reflective elements of the NDRIS are optimized by employing the Riemannian manifold, and the locations of the non-zero entries of the NDRIS phase shift matrix are obtained by the maximal ratio combining (MRC) criterion. Finally, we present our simulation results, which show that deploying an NDRIS achieves additional gains for the CUs over a conventional RIS, further enhancing both the communication efficiency and sensing reliability. Furthermore, we compare the results to the pertinent benchmarks, which validate the effectiveness of our proposed algorithms.},
keywords = {and geometric mean rate, Array signal processing, Base stations, Copper, Costs, Hardware, Integrated sensing and communication, millimeter wave, Millimeter wave communication, Optimization, Radio frequency, reconfigurable intelligent surface, Reconfigurable Intelligent Surfaces},
pubstate = {published},
tppubtype = {article}
}
The joint optimization of the hybrid transmit precoders (HTPCs) and reflective elements of a millimeter wave (mmWave) integrated sensing and communication (ISAC) system is considered. The system also incorporates a reconfigurable intelligent surface (RIS) relying on a non-diagonal RIS (NDRIS) phase shift matrix. Specifically, we consider a hybrid architecture at the ISAC base station (BS) that serves multiple downlink communication users (CUs) via the reflected links from the RIS, while concurrently detecting multiple radar targets (RTs). We formulate an optimization problem that aims for maximizing the geometric mean (GM) rate of the CUs, subject to the sensing requirement for each RT. Additional specifications related to the limited transmit power and unit modulus (UM) constraints for both the HTPCs and the reflective elements of the NDRIS phase shift matrix make the problem challenging. To solve this problem, we first transform the intractable GM rate expression to a tractable weighted sum rate objective and next split the transformed problem into sub-problems. Consequently, we propose an iterative alternating optimization approach that leverages the majorization-minimization (MM) framework and block coordinate descent (BCD) method to solve each sub-problem. Furthermore, to tackle the UM constraints in the sub-problem of the HTPC design, we propose a penalty-based Riemannian manifold optimization (PRMO) algorithm, which optimizes the HTPCs on the Riemannian manifold. Similarly, the phases of the reflective elements of the NDRIS are optimized by employing the Riemannian manifold, and the locations of the non-zero entries of the NDRIS phase shift matrix are obtained by the maximal ratio combining (MRC) criterion. Finally, we present our simulation results, which show that deploying an NDRIS achieves additional gains for the CUs over a conventional RIS, further enhancing both the communication efficiency and sensing reliability. Furthermore, we compare the results to the pertinent benchmarks, which validate the effectiveness of our proposed algorithms.
2.
Qi, Jiaju; Lei, Lei; Jonsson, Thorsteinn; Hanzo, Lajos
Electric Bus Charging Schedules Relying on Real Data-Driven Targets Based on Hierarchical Deep Reinforcement Learning Journal Article
In: IEEE Access, vol. 13, pp. 99415–99433, 2025, ISSN: 2169-3536.
Abstract | Links | BibTeX | Tags: {>}Deep reinforcement learning, Batteries, charging control, Costs, deep reinforcement learning, electric bus, Electricity, hierarchical reinforcement learning, Real-time systems, Schedules, Scheduling, Stochastic processes, Uncertainty, Vehicle-to-grid
@article{qi_electric_2025,
title = {Electric Bus Charging Schedules Relying on Real Data-Driven Targets Based on Hierarchical Deep Reinforcement Learning},
author = {Jiaju Qi and Lei Lei and Thorsteinn Jonsson and Lajos Hanzo},
url = {https://ieeexplore.ieee.org/document/11006647},
doi = {10.1109/ACCESS.2025.3571211},
issn = {2169-3536},
year = {2025},
date = {2025-01-01},
urldate = {2025-10-08},
journal = {IEEE Access},
volume = {13},
pages = {99415–99433},
abstract = {The charging scheduling problem of Electric Buses (EBs) is investigated based on Deep Reinforcement Learning (DRL). A Markov Decision Process (MDP) is conceived, where the time horizon includes multiple charging and operating periods in a day, while each period is further divided into multiple time steps. To overcome the challenge of long-range multi-phase planning with sparse reward, we conceive Hierarchical DRL (HDRL) for decoupling the original MDP into a high-level Semi-MDP (SMDP) and multiple low-level MDPs. The Hierarchical Double Deep Q-Network (HDDQN)-Hindsight Experience Replay (HER) algorithm is proposed for simultaneously solving the decision problems arising at different temporal resolutions. As a result, the high-level agent learns an effective policy for prescribing the charging targets for every charging period, while the low-level agent learns an optimal policy for setting the charging power of every time step within a single charging period, with the aim of minimizing the charging costs while meeting the charging target. It is proved that the flat policy constructed by superimposing the optimal high-level policy and the optimal low-level policy performs as well as the optimal policy of the original MDP. Since jointly learning both levels of policies is challenging due to the non-stationarity of the high-level agent and the sampling inefficiency of the low-level agent, we divide the joint learning process into two phases and exploit our new HER algorithm to manipulate the experience replay buffers for both levels of agents. Numerical experiments are performed with the aid of real-world data to evaluate the performance of the proposed algorithm.},
keywords = {{>}Deep reinforcement learning, Batteries, charging control, Costs, deep reinforcement learning, electric bus, Electricity, hierarchical reinforcement learning, Real-time systems, Schedules, Scheduling, Stochastic processes, Uncertainty, Vehicle-to-grid},
pubstate = {published},
tppubtype = {article}
}
The charging scheduling problem of Electric Buses (EBs) is investigated based on Deep Reinforcement Learning (DRL). A Markov Decision Process (MDP) is conceived, where the time horizon includes multiple charging and operating periods in a day, while each period is further divided into multiple time steps. To overcome the challenge of long-range multi-phase planning with sparse reward, we conceive Hierarchical DRL (HDRL) for decoupling the original MDP into a high-level Semi-MDP (SMDP) and multiple low-level MDPs. The Hierarchical Double Deep Q-Network (HDDQN)-Hindsight Experience Replay (HER) algorithm is proposed for simultaneously solving the decision problems arising at different temporal resolutions. As a result, the high-level agent learns an effective policy for prescribing the charging targets for every charging period, while the low-level agent learns an optimal policy for setting the charging power of every time step within a single charging period, with the aim of minimizing the charging costs while meeting the charging target. It is proved that the flat policy constructed by superimposing the optimal high-level policy and the optimal low-level policy performs as well as the optimal policy of the original MDP. Since jointly learning both levels of policies is challenging due to the non-stationarity of the high-level agent and the sampling inefficiency of the low-level agent, we divide the joint learning process into two phases and exploit our new HER algorithm to manipulate the experience replay buffers for both levels of agents. Numerical experiments are performed with the aid of real-world data to evaluate the performance of the proposed algorithm.