Are you better
than AI?
AI coding agents are ranked on how many real GitHub issues they can fix. Nobody measured the humans. Take the exact same tasks, graded by the exact same tests, and find out where you land.
The test runs on a desktop or laptop. On your phone you can still explore every result.
A bug report from Django, sympy, scikit-learn and friends, picked from the tasks SWE-bench rates as “<15 min fix”.
The whole repository at the original commit: file tree, full-text search, a real editor. Fifteen minutes on the clock.
Your diff runs through the official harness on a public GitHub runner. See which models you beat on exactly the tasks you did.
And humans vs humans
Every task you score counts toward your rank. Fixing an issue most models miss counts for more than one they all solve, so the leaderboard rewards skill, not just stamina.
See the leaderboard →A fair fight, mostly
We kept everything that can be kept identical, and we're upfront about what isn't. Results are reported as they are, whichever side they flatter.
- Tasks
- Real GitHub issues from SWE-bench Verified, at the original commit
- Grading
- The official SWE-bench harness, same Docker images, same hidden tests
- Bar to pass
- Every FAIL_TO_PASS and PASS_TO_PASS test must pass
- Time
- You get 15 minutes. Agents ran until they decided they were done.
- Execution
- You can read, search and edit, but not run code. Agents had a shell.
- Hints
- Neither side sees hints or the tests. Same as the leaderboard.