FineFineWeb Fine-Grained Domain Web Corpus

## Overview FineFineWeb is a large-scale English web corpus from the m-a-p research group, organized into fine-grained subject domains (aerospace, agronomy, artistic, and many more). The collection spans billions of tokens across three iterations of curation, with per-domain token and sample counts reported in the dataset card. Data is provided in tabular/text format suitable for LLM pretraining, domain-specific fine-tuning, and text classification research. Domain coverage is the primary axis of organization, enabling targeted sampling. ## Schema - text — string — raw web document text - domain — string — fine-grained domain label (e.g., aerospace, agronomy, artistic) - iteration — int — curation iteration (1, 2, or 3) the sample was added in - + additional metadata columns per the source dataset card (Schema is inferred from the dataset card summary; consult the HF page for exact column list per shard.) ## Sources - m-a-p/FineFineWeb on Hugging Face — https://huggingface.co/datasets/m-a-p/FineFineWeb — Apache-2.0 ## Methodology Per the m-a-p project description, FineFineWeb is a comprehensive study on fine-grained domain web corpora. The data is sourced from web crawls and then classified/filtered into fine-grained domains across multiple iterations of curation, with each iteration adding additional filtered tokens. Token and sample counts are reported per domain per iteration. Full methodology details (arXiv paper, project page, blog) are listed as "coming soon" by the authors as of the dataset release. ## Known gaps & limitations - English-only; no multilingual coverage. - Web-sourced text inherits crawl biases, duplication risks, and potentially toxic or copyrighted content typical of large web corpora. - Domain classification quality depends on the authors' classifier; mislabeling is possible especially for boundary domains. - Formal paper and methodology documentation were marked "coming soon" at release — buyers should validate empirically rather than rely on a published peer-reviewed pipeline description. - Not deduplicated against common LLM eval suites — leakage risk if used to train models you then benchmark. ## Intended use & out-of-scope - Intended for: domain-aware LLM pretraining and continued pretraining, domain-specific fine-tuning, text classification research, domain RAG corpus construction, and studies of domain composition effects. - Out-of-scope: production use without additional safety/PII filtering; benchmark training without leakage checks; non-English applications. _Federated dataset: 10 parquet shards, 2.72 GB total. Queries and downloads stream through the DataBazaar API._ ## Temporal validity This dataset includes column(s) keyed on recycled identifiers — the same value can refer to different entities at different times: - **domain name** (reissued by registrars (drop-catching)) — domains are recycled; use WHOIS history to filter enrichment by the current registration interval. Original supplier listing: FineFineWeb: Fine-Grained Domain Web Corpus Large-scale (billions of tokens) English web corpus from m-a-p, organized by fine-grained domains (aerospace, agronomy, artistic, etc.) for domain-aware LLM pretraining, classification, and RAG.