James Ding
Feb 18, 2026 20:38
Harvey’s BigLaw Bench Global doubles benchmark size, testing AI legal capabilities across jurisdictions as model scores hit 90% on core tasks.
Harvey AI released BigLaw Bench: Global on February 18, more than doubling its public benchmark dataset with new evaluations for UK, Australia, and Spanish legal systems. The expansion marks the first major update since Harvey announced plans to scale BLB fivefold earlier this month.
The timing matters. Leading foundation models now hit roughly 90% on BLB’s core legal tasks—up from around 60% in 2024. But Harvey’s internal research shows performance degrades when models tackle jurisdiction-specific work. BLB: Global aims to quantify exactly where that localization gap exists.
Six Task Categories Under the Microscope
Harvey built the benchmark around six workflows its enterprise clients actually use: drafting, long document analysis, document comparison, public research, multi-document analysis, and extraction. Each task was designed by local practitioners in collaboration with Mercor, then cross-reviewed by Harvey’s applied legal researchers.
The scenarios get specific. One UK task asks models to advise on FCA enforcement risks when a CSO sells shares before a failed drug trial announcement. A Spanish benchmark involves analyzing CNMC antitrust exposure for tech companies caught in a no-poach agreement. Australian tasks include FIRB approval determinations for infrastructure fund acquisitions.
“The goal of BLB: Global is to help understand and remediate where foundation models struggle to localize effectively on core AI tasks,” Harvey stated in the announcement.
Why This Matters for Enterprise AI Adoption
Law firms operating across borders face a real problem: an AI assistant that handles Delaware corporate law brilliantly might stumble on UK financial regulations or Spanish competition law. Without standardized benchmarks, there’s no way to verify consistent quality across offices.
Harvey’s approach—building jurisdiction-specific tasks with over two dozen local experts—creates a baseline for measuring that consistency. The company plans to extend BLB: Arena, its preference-based evaluation system launched in November 2025, to international markets as well.
More countries are coming. Harvey indicated it will continue building local expert cohorts and deepening existing datasets based on customer feedback. For legal tech buyers evaluating AI vendors, BLB: Global provides something that didn’t exist before: a standardized way to compare model performance on real legal work across multiple jurisdictions.
Image source: Shutterstock
Credit: Source link
