Says Astra for Law passed internal legal research benchmark at higher accuracy

OpenAI validated Astra for Law on a 200‑question private validation set (Vals AI’s Legal Research Bench), reporting a 54% pass rate versus 38.7% for the base model with web search.

In this brief: 2 sections 1 min read

- Evaluated on 200 U.S. legal research questions drawn from Vals AI’s Legal Research Bench.
- Astra for Law passed 54% of questions at the highest reasoning setting.
- GPT‑6 Astra using plain web search passed 38.7%, a roughly 40% relative improvement claimed for Astra for Law.
- OpenAI and partners claim the dedicated index helps reduce made-up citations.
- Performance gains support positioning Astra for Law for AmLaw‑200 firms and legal‑tech builders.
- The evaluation is company‑published; independent verification not cited in this source.

Read full analysis on en.cryptonomist.ch

Useful?