Samaya AI · Finance agent benchmark

FrontierFinance Benchmark Grader

Samaya AI has released a rubrics-based LLM grader for its FrontierFinance benchmark, which measures finance agents. For each query, a panel of judge models decides by majority vote whether each rubric is satisfied by a response you supply, and the tool reports macro and micro qualification rates, including a must-have subset and breakdowns by rubric type and data source type. The recommended judges are Claude Sonnet 4.6, Gemini 3.1 Pro and GPT 5.4, chosen for high agreement with a committee of 9 judges. The rubrics dataset is on Hugging Face under CC-BY-4.0; the grader runs through the official Anthropic, OpenAI and Gemini SDKs.

Open the source ↗

Similar Developer Tools releases

Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.