A LoRA-Based Approach to Fine-Tuning LLMs for Educational Guidance in Resource-Constrained Settings

The current study describes a cost-effective method for adapting large language models(LLMs) for academic advising in study-abroad contexts. Using the Mistral-7B model withLow-Rank Adaptation (LoRA) and 4-bit NF4 quantization via the Unsloth framework, themodel underwent training in two distinct hardware phases to demonstrate adaptability andcomputational efficiency. Contrary to multi-stage data curation, this study utilized a sin-gle synthetic dataset of 2,274 conversation pairs across both phases to evaluate hardware-agnostic convergence. In Phase 1, the model was fine-tuned on an NVIDIA Tesla P100.In Phase 2, the model continued training on the same dataset using an NVIDIA Tesla T4with optimized batch configurations to refine performance. Technical innovations utilizedmemory-efficient quantization and continuous training analytics. After training, the studydemonstrated a total reduction in training loss from ∼1.01 to ∼0.34, achieving stable conver-gence on consumer-grade GPU equipment. These findings support the effective applicationof instruction-tuned LLMs within educational advising, specifically showing that trainingcan be effectively distributed and resumed across varying hardware architectures.