Paraphrase Tremors: Uncovering TinyLLaMA’s Sensitivity to Subtle Rewordings

Natural language instructions can be phrased in countless ways by end users, yet small language models intended for on-device or low-resource deployment may react unpredictably to minor paraphrasing noise. In this paper, we quantify the sensitivity of a 1.1B-parameter chat model (TinyLLaMA-1.1B-Chat) to two independent paraphrases generated by a T5-based paraphrasing strategy. We evaluate response drift across three metrics-embedding cosine similarity, BERTScore, and BLEU-on 80 Alpaca-style instruction prompts. Our results show an average embedding drift of 0.118 (±0.085), with surface-form BLEU drifting by 0.098 and semantic BERTScore-F1 by 0.025. Correlation analysis reveals only a weak link (r=0.127, p=0.263) between prompt variation and response variation. Qualitative failure cases illustrate domain misinterpretation, style oscillation, and task misclassification even when paraphrases remain semantically close. These findings highlight the need for robustness-aware prompt engineering in small-scale LLM deployments.