Amazon Web Services · Model Leaderboard
Amazon SageMaker JumpStart Model Leaderboard
AWS has launched a leaderboard view in SageMaker Studio for comparing JumpStart's open-weight LLMs side by side. At launch it covers 70+ models from providers including Meta, Google, Qwen, IBM and Arcee. Quality is scored on BIG-Bench Hard, GPQA Diamond and MMLU-Pro, run by AWS with one shared harness, while speed (time to first token, tokens per second) and instance cost are measured on SageMaker endpoints for each deployment configuration. Cost figures cover SageMaker instance cost only.
Open the source ↗Similar Developer Tools releases
- AI data preparation capabilities in Amazon SageMaker
Amazon Web Services · Data Prep Toolkit · Week 48, 2026
- Catching JiTTests
Meta · LLM Test Generation · Week 48, 2026
- NVIDIA Vera Rubin NVL72 in MLPerf Training v6.1
NVIDIA · Training Benchmark Results · Week 48, 2026
- FrontierFinance Benchmark Grader
Samaya AI · Finance agent benchmark · Week 47, 2026
- AWS Emulator
AWS · Agent-built AWS emulator · Week 47, 2026
- UAEF (Universal Agent Evaluation Framework)
AWS Machine Learning · Agent Eval Framework · Week 47, 2026
More from Amazon Web Services
- Workload recovery and infrastructure checks on Amazon SageMaker HyperPod
Amazon Web Services · Training Health Checks · Week 48, 2026
- Custom metadata schemas in AWS Agent Registry
Amazon Web Services · Agent Registry Metadata · Week 48, 2026
- AI data preparation capabilities in Amazon SageMaker
Amazon Web Services · Data Prep Toolkit · Week 48, 2026
- Retrieval instructions for Amazon Bedrock Managed Knowledge Base
Amazon Web Services · Per-tenant RAG · Week 48, 2026
- Tabular search in Amazon Bedrock Managed Knowledge Base
Amazon Web Services · Table Agentic Retrieval · Week 48, 2026
- SageMaker HyperPod Data Capture
Amazon Web Services · Inference Data Capture · Week 47, 2026
Published by Quanta AI Labs, an AI agency in India. Browse every week in AI News Weekly.