FedTLRec: Federated Recommendation with Transformer-based Parameter Aggregation and LoRA Compression

Federated learning has emerged as a key paradigm in privacy-preserving computing due to its "data usable but not visible" property, enabling users to collaboratively train models without sharing raw data. Motivated by this, federated recommendation systems offer a promising architecture that balances user privacy with recommendation accuracy through distributed collaborative learning. However, existing federated recommendation systems face significant challenges in balancing model performance, communication efficiency, and user privacy. In this paper, we propose FedTLRec (Federated Recommendation with Transformer-based Parameter Aggregation and Collaborative LoRA), a novel federated recommendation framework that combines Low-Rank Adaptation (LoRA) parameter compression with Transformer-based parameter aggregation. Our approach addresses the communication bottleneck in federated learning by employing LoRA to compress client model updates, which significantly reduces data transmission overhead. Additionally, we introduce a Transformer-based aggregation model on the server side to integrate parameter updates from multiple clients effectively. This model leverages attention mechanisms to capture relationships between clients. To further enhance efficiency, we implement a K-means clustering strategy to group clients with similar characteristics before the parameter aggregation process. Extensive experiments conducted on real-world datasets demonstrate that FedTLRec achieves superior recommendation performance while substantially reducing communication costs compared to state-of-the-art federated recommendation methods. Furthermore, our method effectively manages client dropout scenarios, maintaining robust performance even when a portion of clients are offline. The code will be open sourced https://github.com/trueWangSyutung/Kmean-Lora-PFedRec