Joint Beamforming for RIS Assisted ISAC Systems with Discrete Phase Shifts: A DRL Based Approach

Integrated sensing and communication (ISAC) is regarded as a promising approach to alleviate spectrum congestion in future communication networks. Leveraging recent breakthroughs in programmable meta-materials, re-configurable intelligent surfaces (RIS) enhance communication systems by enabling multiple input multiple output (MIMO) transmission without requiring additional radio frequency (RF) chains. Therefore, integrating RIS into ISAC systems presents a promising avenue to address the fundamental challenge of enhancing ISAC system performance through only active beamforming, especially in adverse channel conditions. This correspondence investigates the joint optimization of transmit beamforming and RIS reflection coefficients to maximize the sum capacity of ISAC systems. The study adopts a more practical approach by constraining each RIS element to use only a discrete set of phase shifts. Resulting non-convex optimization problem has been addressed using the deep deterministic policy gradient (DDPG) and the twin delayed deep deterministic (TD3) policy gradient deep reinforcement learning (DRL) algorithms. The numerical results and analysis demonstrate the superior performance of TD3 compared to DDPG in terms of the improved sum capacity of ISAC system.