Sparsity-Aware Edge Caching in IoVs with Asynchronous Federated and Deep Reinforcement Learning

The edge content caching technology of the Internet of Vehicles (IoVs) is a key technology to reduce the latency of content access. However, within the hotspot area, with a large number of content access requests generated by vehicle users, the rapid changes in user interests and the explosive dissemination of high-value content have led to limited transmission delay. Therefore, to reduce the delay, accurately predicting and timely updating popular content as well as exploring high-value content have become critical yet challenging. To solve this problem, a sparsity-aware edge caching (SAEC) scheme is proposed. Firstly, aiming at the sparsity problem of VU data, a sparse self-encoder based on self-attentive (SAE-ELA) model is proposed. By extracting the potential features of sparse data of vehicle users and capturing the historical preference associations of users, the accuracy of content prediction was improved. Secondly, this paper adopts the asynchronous federated learning (AFL) framework to solve the problem of low cache update efficiency, thereby shortening the model training time to improve the real-time performance of content update. Finally, in order to solve the problem of insufficient exploration of potential high-value content, a Dueling Deep Q-network based on Intrinsic Curiosity Module (ICM-DDQN) algorithm is proposed. By organically combining traditional value function learning with curiosity driven active exploration, the exploration efficiency of cached content has been improved, thereby reducing the Content Transmission Delay (CTD). Simulation results show that the proposed SAEC is significantly superior to the existing methods in terms of Cache Hit Ratio(CHR) and CTD.