Evaluation of Large Language Models: Review of Metrics, Applications, and Methodologies

Large Language Models (LLMs) have revolutionized various domains, including finance, medicine, and education. This review paper provides a comprehensive survey of the key metrics and methodologies employed to evaluate LLMs. We discuss the importance of evaluation, explore a wide range of metrics covering aspects such as accuracy, coherence, relevance, and safety, and examine different evaluation frameworks and techniques. We also address the challenges in LLM evaluation and highlight best practices for ensuring reliable and trustworthy AI systems. This survey draws upon a wide range of recent research and practical insights to offer a holistic view of the current state of LLM evaluation.We surveyed a comprehensive evaluation framework integrating quantitative metrics like entropy-based stability measures and domain-specific scoring systems for medical diagnostics and financial analysis, while addressing persistent challenges including hallucination rates (28\% of outputs from current research) and geographical biases in model responses. The study proposes standardized benchmarks and hybrid human-AI evaluation pipelines to enhance reliability, supported by algorithmic innovations in training protocols and RAG architectures. Our findings underscore the necessity of robust, domain-adapted evaluation methodologies to ensure the safe deployment of LLMs in high-stakes applications. Through systematic analysis of 70+ studies, this paper revisits that while LLMs achieve near-human performance in structured tasks like certifications exams, they exhibit critical limitations in open-ended reasoning and output consistency. Our analysis covers foundational concepts in prompt engineering, evaluation methodologies from industry and academia, and practical tools for implementing these assessments. The paper examines key challenges in LLM evaluation, including bias detection, hallucination measurement, and context retention, while proposing standardized approaches for comparative analysis. We demonstrate how different evaluation frameworks can be applied across domains such as technical documentation, creative writing, and factual question answering. The findings provide practitioners with a structured approach to selecting appropriate evaluation metrics based on use case requirements, model characteristics, and desired outcomes.