Vals AI test finds models lag on referrals to local legal aid
A Vals AI and NextLadder Ventures benchmark of 23 models across 252 scenarios found that general legal explanations are materially stronger than locating specific local help.
AT A GLANCE
Key points
- The test evaluated 23 models on 252 expert-written legal-aid scenarios.
- No model crossed the 80% reliability threshold.
- Scores ranged from 78–87% for general explanations and 30–50% for specific local resources.
- Estimated free-tier and premium averages were close.
What is changing?
Vals AI and NextLadder Ventures released the Access to Justice Benchmark, evaluating 23 models on 252 scenarios written by legal-aid experts in six states. Qwen 3.8 Max ranked first at 76.37%, and no model crossed the benchmark’s 80% threshold for meeting expert rubric criteria. The report says the differences among the top three are within the margin of error.
Models met 78–87% of criteria for general legal explanations but only 30–50% for identifying a specific local legal-aid organization or court self-help center. Twenty-one of 23 models used national directories in at least 70% of their final answers. Vals also reported a 69.7% average for estimated free-tier models and 70.4% for premium models; that tier mapping is the benchmark team’s estimate, not a provider-confirmed assignment.
Explore the original sources
Source date: 10 November 2026.
Published by AI Radar: 11 Nov 2026, 00:12 (Istanbul, UTC+3).
FOLLOW THE TOPIC
Topics and tags
Share your discovery.
Send this story to a friend.
Reader comments
What do you think about this development? Share your experience or question. Comments are published after editorial approval.
Your name/nickname and comment become public if approved. Read: Privacy notice.
Loading comments…