Adversarial attack detection in resource-constrained environments: A stable and sequential federated learning architecture with TinyLlama-1.1B

Large Language Models (LLMs) face training challenges on resource-constrained devices, and the performance losses caused by compression methods necessitate a ‘full fine-tuning’ approach. In this study, a Mutex-based architecture is proposed for the full fine-tuning of the TinyLlama-1.1B model in a federated learning environment. The proposed method prevents Out of Memory (OOM) errors by queuing GPU access while minimizing system load and providing an efficient training process through ‘Incremental Averaging’ and FP16 optimization on the server side.In experiments conducted on 5 clients using the TCAB dataset, the conventional federated learning method failed due to memory constraints, while the proposed method completed training with a 100 % success rate. As a result of the training, the model's accuracy in detecting adversarial attacks increased from 60.84% to 99.02%, and the balanced distribution of Precision and Recall values proved that the model did not develop bias. Additionally, FP16 optimization on the server side resulted in a 3.2 GB memory savings and reduced server computation costs. The findings reveal that despite increasing the total training time (latency), the proposed architecture enables large language models to be trained securely and stably with full fine-tuning, even on resource-constrained edge devices.