Datacurve · Evaluation & Benchmarks

DeepSWE

Coding-agent benchmark of 113 hand-authored long-horizon tasks across 91 repos and 5 languages (TS, Go, Python, JS, Rust) using Harbor format with Docker envs and behavioral verifiers (0.3% FP / 1.1% FN). Hand-written from scratch to avoid pretraining contamination — current leader gpt-5.5[xhigh] hits 70% solve rate.