Artificial Intelligence and Law, 2011–2026: A Systematic Scoping Review of Methods, Benchmarks, and Open Challenges

This systematic scoping review examines how research at the intersection of Artificial Intelligence and Law (AI & Law) has evolved over the fifteen-year period from 2011 to 2026. Following a PRISMA-ScR-informed protocol, we synthesise contributions published primarily in Artificial Intelligence and Law and related venues across two converging paradigms: (i) symbolic, argumentation-based, and formal models for legal knowledge representation, normative reasoning, and justification, and (ii) statistical, machine learning, and natural language processing (NLP) approaches that analyse, predict, and retrieve legal text at scale. Our core finding is that the field has transitioned from a dichotomy of 'AI or Law' toward hybrid socio-technical systems in which formal guarantees—normative consistency, traceability, and human oversight—must coexist with empirical performance demands such as robust generalisation, reproducibility, and realistic task evaluation. Methodologically, a clear shift from relatively closed, domain-specific systems toward open benchmarks, open data, and open implementations is observable, particularly in legal NLP and legal information retrieval/entailment competitions. Yet a crucial distinction persists: the difference between 'predicting correctly' and 'reasoning legally.' Multiple contributions emphasise that predictive models without adequate explanation and justification frameworks remain legally and socially problematic. We operationalise a triadic taxonomy—text-centric, reasoning-centric, and governance-centric—and map representative works onto method families (symbolic, statistical, hybrid), datasets and benchmarks, and application domains (contract analysis, e-discovery, compliance checking, adjudication support, and argument mining). The EU AI Act's risk-based framework, with phased applicability through 2026–2027, directly amplifies research questions around transparency, documentation, human oversight, and data quality. We conclude with a concrete research agenda identifying five open challenges: justification-oriented benchmark design, hybrid LLM-plus-formal-constraint architectures, multilingual and cross-jurisdiction transfer, human-centred evaluation protocols, and open-texture detection in regulatory text.