DeepSearchQA Agent Factuality Benchmark
Updated Oct 10, 2026
Source, license and coverage
Supplier documentation. These claims are separate from the automated sample score. A listing edit date is not a data freshness date.
- License
- apache-2.0
- Source / creator
- google/deepsearchqa
- Collection method
- Per Google DeepMind, prompts were authored to require difficult multi-step information seeking across 17 fields, going beyond traditional single-document QA. The benchmark targets "deep research" agent capabilities — comprehensiveness, multi-hop retrieval, and factuality of synthesized answers. Evaluation methodology and grading rubric are documented in the accompanying technical report and starter code; this listing distributes the prompt set itself.
- Coverage start
- Not documented
- Coverage end
- Not documented
- Data last updated
- Not documented
- Update schedule
- Not documented
- English-only; no multilingual coverage. - Small size (900 prompts) — intended as an eval set, not a training corpus. - As a public benchmark, contamination risk is non-trivial: prompts may already appear in web crawls used for LLM pretraining, so frontier models may have partial exposure. - Field distribution across the 17 domains is not guaranteed to be balanced; buyers should inspect before drawing per-domain conclusions. - Source does not exhaustively document annotator demographics or prompt-authoring process beyond the technical report.
Sample structure score: 99.3 / 100
This automated check describes the inspected sample, not factual accuracy, legal rights, representativeness, or the quality of the entire dataset. It is not a customer rating.
Assessed 10 sample records (CSV) on 2026-10-10. All records in the provided sample were checked.
| Check | Points | Evidence |
|---|---|---|
| Populated cells | 50 / 50 | 40 of 40 top-level cells contain a value. Null, absent and blank values count as missing; zero and false count as populated. |
| Consistent value types | 29.3 / 30 | 39 of 40 populated cells match their column's most common observed type. Types are inferred, not checked against real-world truth. |
| Consistent record shape | 20 / 20 | 10 of 10 records have the expected fields. CSV/TSV use the header width; JSON uses the union of observed keys. |
Field-level findings and improvements
Check missing cells and mixed types below. Document intentional missing values or mixed types in your field descriptions. Do not fill legitimate unknowns with invented values just to increase this score.
| Field | Missing cells | Most common type | Other populated types |
|---|---|---|---|
| problem | 0 / 10 | string | 0 / 10 |
| problem_category | 0 / 10 | string | 0 / 10 |
| answer | 0 / 10 | string | 1 / 10 |
| answer_type | 0 / 10 | string | 0 / 10 |
About this data
Multi-step information-seeking benchmark spanning 17 fields for evaluating agent factuality and reasoning on complex retrieval tasks.
Retrieve with your agent or Python
Export DeepSearchQA to CSV for an agent evaluation →
Create an account and configure DATABAZAAR_API_KEY. This example retrieves free or already purchased data; it never makes a purchase. For a multi-file dataset, choose a file index from the manifest.
Download the Python examplepython3 retrieve-dataset.py 24cf80b3-5f47-4da6-a6a5-0540b3688343 --output dataset.bin
Schema
| Name | Type | Description |
|---|---|---|
| problem | string | |
| problem_category | string | |
| answer | string | |
| answer_type | string |
Sample Data
Preview a sample of the data before downloading.
Public sample only. Sign in to retrieve the full dataset, including free datasets.
For AI Agents
# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
"mcpServers": {
"databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
}
}
# 2. Your agent can then call:
search_datasets({ query: "DeepSearchQA Agent Factuality " })
// Found: 24cf80b3-5f47-4da6-a6a5-0540b3688343
get_download_url({ dataset_id: "24cf80b3-5f47-4da6-a6a5-0540b3688343" }) // free — sign in with MCP OAuth first# Free dataset — sign in or use your account API key: curl https://api.databazaar.io/datasets/24cf80b3-5f47-4da6-a6a5-0540b3688343/download-url -H "Authorization: Bearer $DATABAZAAR_API_KEY"