SmolLM Corpus Pretraining Data
Source, license and coverage
Supplier documentation. These claims are separate from the automated sample score. A listing edit date is not a data freshness date.
- License
- odc-by
- Source / creator
- HuggingFaceTB/smollm-corpus
- Collection method
- Cosmopedia v2 was synthetically generated by prompting Mixtral-8x7B-Instruct with curated seed topics drawn from web data and educational resources, producing ~39M textbook/blog/story-style documents across many audiences and formats. FineWeb-Edu-dedup was produced by running an educational-quality classifier over FineWeb (a CommonCrawl-derived web corpus) and keeping the high-scoring subset, then deduplicating. Python-Edu was filtered from The Stack using an educational-code classifier. The publisher applied quality filtering, deduplication, and language identification before release.
- Coverage start
- Not documented
- Coverage end
- Not documented
- Data last updated
- Not documented
- Update schedule
- Not documented
- English-only; not suitable for multilingual model training. - Cosmopedia v2 is fully synthetic — it inherits any factual errors, hallucinations, or stylistic biases of Mixtral-8x7B-Instruct, and is not a substitute for ground-truth knowledge sources. - FineWeb-Edu subset inherits the biases of CommonCrawl (overrepresentation of certain web demographics and topics) and of the educational-quality classifier. - Not deduplicated against standard LLM evaluation suites — leakage risk if used for benchmark training. - Source does not document demographic/topical balance audits beyond classifier scores; buyers should validate empirically for their use case. The supplier describes synthetic or modeled records. These should not be treated as verified real-world observations.
Sample structure score: 100 / 100
This automated check describes the inspected sample, not factual accuracy, legal rights, representativeness, or the quality of the entire dataset. It is not a customer rating.
Assessed 10 sample records (JSON) on 2026-10-09. All records in the provided sample were checked.
| Check | Points | Evidence |
|---|---|---|
| Populated cells | 50 / 50 | 60 of 60 top-level cells contain a value. Null, absent and blank values count as missing; zero and false count as populated. |
| Consistent value types | 30 / 30 | 60 of 60 populated cells match their column's most common observed type. Types are inferred, not checked against real-world truth. |
| Consistent record shape | 20 / 20 | 10 of 10 records have the expected fields. CSV/TSV use the header width; JSON uses the union of observed keys. |
Field-level findings and improvements
Check missing cells and mixed types below. Document intentional missing values or mixed types in your field descriptions. Do not fill legitimate unknowns with invented values just to increase this score.
| Field | Missing cells | Most common type | Other populated types |
|---|---|---|---|
| blob_id | 0 / 10 | string | 0 / 10 |
| repo_name | 0 / 10 | string | 0 / 10 |
| path | 0 / 10 | string | 0 / 10 |
| length_bytes | 0 / 10 | number | 0 / 10 |
| score | 0 / 10 | number | 0 / 10 |
| int_score | 0 / 10 | number | 0 / 10 |
About this data
Educational and synthetic pretraining corpus combining Cosmopedia v2 synthetic textbooks and stories with Python-Edu and deduplicated FineWeb-Edu subsets, curated for small language model training.
Retrieve with your agent or Python
Create an account and configure DATABAZAAR_API_KEY. This example retrieves free or already purchased data; it never makes a purchase. For a multi-file dataset, choose a file index from the manifest.
Download the Python examplepython3 retrieve-dataset.py 61901d41-9eaf-4463-8dbf-79684b69fc7d --output dataset.bin
Full supplier documentation
Schema
| Name | Type | Description |
|---|---|---|
| blob_id | VARCHAR | SHA-1 hash uniquely identifying the code file |
| repo_name | VARCHAR | GitHub repository name containing the source code file |
| path | VARCHAR | File path within the repository |
| length_bytes | BIGINT | File size in bytes |
| score | DOUBLE | Quality score (0–5 range, float) for educational value |
| int_score | BIGINT | Quality score rounded to nearest integer (1–5) |
Sample Data
Preview a sample of the data before downloading.
Public sample only. Sign in to retrieve the full dataset, including free datasets.
For AI Agents
# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
"mcpServers": {
"databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
}
}
# 2. Your agent can then call:
search_datasets({ query: "SmolLM Corpus Pretraining Data" })
// Found: 61901d41-9eaf-4463-8dbf-79684b69fc7d
get_download_url({ dataset_id: "61901d41-9eaf-4463-8dbf-79684b69fc7d" }) // free — sign in with MCP OAuth first# Free dataset — sign in or use your account API key: curl https://api.databazaar.io/datasets/61901d41-9eaf-4463-8dbf-79684b69fc7d/download-url -H "Authorization: Bearer $DATABAZAAR_API_KEY"