otherScaleAI/SWE-bench_Proswe-benchcoding-agentsbenchmarkevaluationsoftware-engineeringscale-aiagentslong-horizoncodellm-eval
SWE-bench Pro
About this data
Enterprise-level benchmark dataset from Scale AI for evaluating AI agents on long-horizon software engineering tasks. Follows SWE-Bench Verified structure with challenging real-world coding problems.
Schema
| Name | Type | Description |
|---|---|---|
| repo | VARCHAR | |
| instance_id | VARCHAR | |
| base_commit | VARCHAR | |
| patch | VARCHAR | |
| test_patch | VARCHAR | |
| problem_statement | VARCHAR | |
| requirements | VARCHAR | |
| interface | VARCHAR | |
| repo_language | VARCHAR | |
| fail_to_pass | VARCHAR | |
| pass_to_pass | VARCHAR | |
| issue_specificity | VARCHAR | |
| issue_categories | VARCHAR | |
| before_repo_set_cmd | VARCHAR | |
| selected_test_files_to_run | VARCHAR | |
| dockerhub_tag | VARCHAR |
Sample Data
Preview a sample of the data before downloading.
Free
Open dataset
Quality: No ratings
1 downloads
Seller: DataBazaar
Agent? No sign-up needed →
For AI Agents
Via MCP Server
# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
"mcpServers": {
"databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
}
}
# 2. Your agent can then call:
search_datasets({ query: "SWE-bench Pro" })
// Found: de2141e6-1a79-421b-a389-801739457e65
get_download_url({ dataset_id: "de2141e6-1a79-421b-a389-801739457e65" }) // free — no API key neededVia REST API
# Free dataset — no API key required: curl https://api.databazaar.io/datasets/de2141e6-1a79-421b-a389-801739457e65/download-url