textopen-web-math/open-web-mathmathematicsllm-pretrainingcommon-crawlreasoninglatexfine-tuningnlpopen-data

OpenWebMath Corpus

Free

Open dataset

Sample structure: 100 / 100

2 download links issued

Seller: DataBazaar

Already have an account? Log in

Agent? Connect your account →

Source, license and coverage

Supplier documentation. These claims are separate from the automated sample score. A listing edit date is not a data freshness date.

License: ODC-By-1.0
Source / creator: open-web-math/open-web-math
Collection method: The authors processed over 200B HTML documents from Common Crawl using a custom pipeline that prioritizes correct extraction of mathematical content (including LaTeX, MathML, and inline math). The pipeline includes math-aware HTML extraction, language filtering (English), perplexity-based quality filtering using a KenLM model trained on ProofPile, MathScore classification, and near-duplicate removal via MinHash LSH. The result is a curated set of documents enriched for mathematical reasoning content.
Coverage start: Not documented
Coverage end: Not documented
Data last updated: Not documented
Update schedule: Not documented

Source documentation ↗

License terms ↗

The dataset is English-only and Common Crawl-derived, inheriting Common Crawl's biases toward popular and indexable web content. Mathematical notation extraction quality varies by source page structure. The dataset is static (Oct 2023 snapshot) and does not include newer web content. It has not been explicitly decontaminated against all common math benchmarks (MATH, GSM8K, etc.) — buyers training on it for benchmark evaluation should perform their own decontamination. Source README requires compliance with Common Crawl terms and preserves underlying content licenses.

Sample structure score: 100 / 100

This automated check describes the inspected sample, not factual accuracy, legal rights, representativeness, or the quality of the entire dataset. It is not a customer rating.

Assessed 10 sample records (JSON) on 2026-10-09. All records in the provided sample were checked.

Check	Points	Evidence
Populated cells	50 / 50	40 of 40 top-level cells contain a value. Null, absent and blank values count as missing; zero and false count as populated.
Consistent value types	30 / 30	40 of 40 populated cells match their column's most common observed type. Types are inferred, not checked against real-world truth.
Consistent record shape	20 / 20	10 of 10 records have the expected fields. CSV/TSV use the header width; JSON uses the union of observed keys.

Field-level findings and improvements

Check missing cells and mixed types below. Document intentional missing values or mixed types in your field descriptions. Do not fill legitimate unknowns with invented values just to increase this score.

Field	Missing cells	Most common type	Other populated types
url	0 / 10	string	0 / 10
text	0 / 10	string	0 / 10
date	0 / 10	string	0 / 10
metadata	0 / 10	string	0 / 10

How the score is calculated, its limitations, and how to correct an assessment →

About this data

Mathematical documents filtered from Common Crawl, suitable for LLM math pretraining and fine-tuning. Includes diverse mathematical web text across multiple domains.

Retrieve with your agent or Python

Create an account and configure DATABAZAAR_API_KEY. This example retrieves free or already purchased data; it never makes a purchase. For a multi-file dataset, choose a file index from the manifest.

Download the Python example

python3 retrieve-dataset.py f7dfef63-7d71-4a74-b961-22c69302629b --output dataset.bin

Full supplier documentation

## Overview OpenWebMath is a large-scale dataset of mathematical text extracted from the web, comprising approximately 6.3 million documents and 14.7 billion tokens. The data was filtered from over 200 billion HTML files in Common Crawl, with careful extraction of mathematical content including LaTeX equations. Released October 2023 in Parquet format, it is designed for pretraining and fine-tuning large language models on mathematical reasoning. ## Schema - `url` — string — source URL of the document - `text` — string — extracted mathematical text content with preserved LaTeX - `date` — string — crawl/publication date - `metadata` — string/json — additional document metadata (extraction info, math score, etc.) ## Sources - OpenWebMath on HuggingFace: https://huggingface.co/datasets/open-web-math/open-web-math — license: ODC-By 1.0 - Paper: https://arxiv.org/abs/2310.06786 - GitHub: https://github.com/keirp/OpenWebMath ## Methodology The authors processed over 200B HTML documents from Common Crawl using a custom pipeline that prioritizes correct extraction of mathematical content (including LaTeX, MathML, and inline math). The pipeline includes math-aware HTML extraction, language filtering (English), perplexity-based quality filtering using a KenLM model trained on ProofPile, MathScore classification, and near-duplicate removal via MinHash LSH. The result is a curated set of documents enriched for mathematical reasoning content. ## Known gaps & limitations The dataset is English-only and Common Crawl-derived, inheriting Common Crawl's biases toward popular and indexable web content. Mathematical notation extraction quality varies by source page structure. The dataset is static (Oct 2023 snapshot) and does not include newer web content. It has not been explicitly decontaminated against all common math benchmarks (MATH, GSM8K, etc.) — buyers training on it for benchmark evaluation should perform their own decontamination. ## Intended use & out-of-scope - IS for: pretraining and fine-tuning LLMs on mathematical reasoning, building math-focused RAG corpora, research on math language modeling, training math-specialized base models (as done in Llemma, DeepSeekMath, etc.). - NOT for: production math tutoring without verification (web content quality varies), benchmark training without decontamination checks, non-English math applications. _Federated dataset: 114 parquet shards, 25.55 GB total. Queries and downloads stream through the DataBazaar API._ _PII signals: us_phone×1, credit_card_candidate×1 (0.3≤score<0.7) present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ Original supplier listing: OpenWebMath: 14.7B Tokens of Mathematical Web Text 6.3M high-quality mathematical documents (14.7B tokens) filtered from 200B+ Common Crawl HTML files. Parquet format, ideal for LLM math pretraining and fine-tuning.

Schema

Name	Type	Description
url	VARCHAR	string — source URL of the document
text	VARCHAR	string — extracted mathematical text content with preserved LaTeX
date	VARCHAR	string — crawl/publication date
metadata	VARCHAR	string/json — additional document metadata (extraction info, math score, etc.)

Sample Data

Preview a sample of the data before downloading.

Public sample only. Sign in to retrieve the full dataset, including free datasets.

For AI Agents

Via MCP Server

# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
  "mcpServers": {
    "databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
  }
}

# 2. Your agent can then call:
search_datasets({ query: "OpenWebMath Corpus" })
// Found: f7dfef63-7d71-4a74-b961-22c69302629b
get_download_url({ dataset_id: "f7dfef63-7d71-4a74-b961-22c69302629b" })  // free — sign in with MCP OAuth first

Via REST API

# Free dataset — sign in or use your account API key:
curl https://api.databazaar.io/datasets/f7dfef63-7d71-4a74-b961-22c69302629b/download-url -H "Authorization: Bearer $DATABAZAAR_API_KEY"