How Frontier AI Models Perform on Real Legal Work

A legal-only bench marking study designed and graded by qualified attorneys across various legal task types

Executive Summary

Percipient designed and administered a structured benchmark to measure how leading large language models perform on the work of lawyers and legal professionals. The study was loosely based on the GDPVAL evaluation framework and adapted for the specific requirements of legal practice, where reasoning, citation accuracy, and faithful adherence to controlling authority matter.

The benchmark exercise included legal task types across various practice areas: litigation, contract redlining, employment, and insurance coverage. Each task asks the model to produce an actual deliverable a lawyer would deliver to a client or senior partner: a memo, a coverage opinion, a redlined contract, or a coded document review set with a privilege log.

Outputs from frontier models across Anthropic, OpenAI, Google, xAI, Moonshot AI, and DeepSeek were collected and blindly graded by qualified legal professionals using detailed, pre-determined rubrics. The results show meaningful and consistent differentiation between models, with reasoning-focused and extended-thinking variants outperforming standard configurations on the most demanding analytical work.

This article describes the study design, methodology, scoring framework, models evaluated, and the results.

Why a Legal AI Benchmark?

Many public benchmarks for artificial intelligence models may not optimally grade how AI performs on legal work. This is so because the LLM benchmarking exercises are often combined with other disciplines, are self graded or rely on bar-exam style questions.

While these benchmarking exercises are thoughtful and valuable, the nuance of true legal work is hard to master and measure. A practicing lawyer is not asked to choose between four multiple choice options. Lawyers are asked to read a complaint, an insurance policy or employment file, identify the issues that matter and apply the controlling authority to produce a deliverable that a client or senior partner may rely on. Whether a model can do this, and how reliably, is the question Percipient set out to answer.

The benchmark design was based on four pillars:

• • Real-world legal issues. Every task is grounded in a realistic fact pattern, with reference materials a practicing lawyer would actually use: insurance policies, contract playbooks, employment files, and binding case law. Material for each task also includes deliberate noise: facts or documents that look topically relevant but are not germane to the legal task. This tests whether a model is applying genuine legal judgment or pattern-matching on surface features.

• • Multimodal deliverables. For the benchmarking exercise. the models are not asked to just summarize legal documents or provide high level legal analysis. The benchmark prompts require an output a lawyer would actually deliver–a coverage opinion, an employment law memo, a coded document review with privilege log, or a redlined contract with comments. Accurate format is part of the score, because in practice a messy deliverable has diminished value.

• • Blinded, rubric-based grading. Rubrics were created by legal professionals and finalized before output review. Graders scored each deliverable without knowing which model produced it. Where two graders scored the same set, their passes are kept independent and reconciled only after both were complete.

• • Qualified legal professionals. All model output and deliverables were graded by attorneys with an average of over 25 years of legal experience. Domain experts were used for each corresponding area of law.

The Legal Tasks

Each task includes a structured prompt, supporting reference materials, and a defined deliverable format. The legal tasks are summarized below.

Models Evaluated

The evaluation measured models from Frontier AI companies. As each task was administered, the then-current production version of each model was used. Where extended-thinking or reasoning variants were available, they were included alongside their standard counterparts.

The employment and contract review benchmarks, which were administered after the coverage and document review work, used updated model versions. The master dataset preserves these distinctions in a private key so that grading remains blind by memo number and is not biased by knowledge of which specific model produced a given output.

Methodology

Evaluation methodology followed a structured workflow designed to minimize bias at every stage.

Grading Framework

Each rubric was task-specific and used partial-credit scoring at the sub-task level. Sub-task grading roll up into section subtotals, and section subtotals roll up into a 100-point overall total. Rubrics were anchored to specific authority and specific facts in each prompt so that grading reflects whether the model did the legal work the task required, not whether the writing was generally sound.

Coverage Rubric

Employment Rubric

Document Review Rubric

Contract Review

Model Results

Coverage

Document Review

Employment

Contract Review

The completed tasks support several preliminary observations.

• • Models cluster tightly on routine work and deviate on analytical work. On document review, nine of ten model variants scored between 89 and 97 out of 100. On the coverage memo, the same model families spread across a 37-point range. The harder the task, the more the rubric distinguishes between systems that genuinely apply controlling authority and systems that merely sound legally fluent.

• • Reasoning-mode models lead the analytical tasks. Across the coverage, employment, and contract benchmarks, tasks that require sustained multi-step legal reasoning, the top scorers were consistently extended-thinking or reasoning-mode variants. Claude Opus 4.7 (Adaptive) and GPT 5.5 Thinking were the top two on contract review and tied for the top score on employment, and Claude Opus 4.7 (Adaptive) also led the coverage benchmark.

• • Extended thinking helps where the analysis is hard, not where the work is large. The extended-thinking variant of Claude Opus 4.6 was the top scorer on document review, but standard variants of the same family were within roughly three points. On the harder analytical tasks, the gap opens up and reasoning-mode variants pulled away from their standard counterparts.

• • Format and writing are not free points. Across both expert passes on the coverage benchmark, the lowest-scoring model lost roughly half of its points in the legal-accuracy and writing sections combined — driven by hallucinated authority, irrelevant arguments, and a deliverable that did not satisfy the memo specification. On the contract task, format alone is worth twenty points, and the lowest format scores came from models that produced substantive analysis but failed to generate a usable redlined .docx or properly attached inline comments.

• • Two-grader passes are worth the cost. On the coverage benchmark, the average difference between Expert 1 and Expert 2 scores was about 11 points; on contract review, the average gap was about 6 points but several individual outputs differed by more than 15 points. Both pairs of passes nonetheless agreed on the top of the field, suggesting the rubrics are doing real work — but it also suggests that a single-grader pass would have produced a less reliable ordering in the middle of the field.

• • Noise resistance is a real differentiator. Each task includes deliberate noise, facts and documents that look topically relevant but are not legally responsive. On the employment memo, where the fact pattern includes both hallucination traps and a Dark v. Curry County noise test, the spread between the highest and lowest scorers on the Quality Control section was meaningful, even though the rubric weighting on that section is only fifteen points.

How These Results Should Be Used

This benchmark is not a vendor scorecard. It does not declare a winner. In the real world, models change, prompts vary, and the same model can perform differently on fact patterns. The point of the work is methodological: to show what a defensible legal AI evaluation looks like, to provide a repeatable framework for measuring real legal performance, and to give legal teams a basis for asking sharper questions when they evaluate a tool.

Ultimately, the most critical takeaway from this benchmark isn’t the final score, but the fact that veteran attorneys with decades of domain expertise still varied when grading complex legal problems. This proves that true legal work is deeply nuanced, subjective, and there is still much room for improvement with model outputs. 

The full data set may be found on Hugging Face.

Additional Suggested Reading: What Does it Really Cost to Use AI for Legal Work?