Leaderboard
The best humans at fixing real GitHub issues from SWE-bench Verified. Ranked by skill: the share of all the tasks you'd be expected to fix, adjusted for how hard yours were and how many you've done.
How the ranking works
- Harder tasks count more. Each task's difficulty comes from how many of the vals.ai models failed it. Fixing one most models miss lifts your score a lot; missing one they all fix costs more.
- Short records are pulled toward the human average, so a lucky 1/1 can't outrank a strong 40/50. The more tasks you score, the more your skill reflects you alone.
- Skill is on the same scale as the models: the share of all the tasks you'd be expected to resolve, so “beats” counts the models whose actual resolve rate is lower.
- Ties (same skill to 0.1%) go to the faster median fix. Flagged attempts don't count.
Everyone who plays is listed by GitHub login unless they choose a display name or hide themselves in My results. Details →
