Dolci Instruct SFT Mixture

## Overview The Dolci Instruct SFT Mixture is the supervised fine-tuning dataset used by AI2 to train Olmo 3 7B Instruct SFT. It contains 2,152,112 instruction-response samples in Parquet format, drawn from a curated mixture of existing prompt sets, covering 70+ languages. Released February 2026. ## Schema - prompt — string — user instruction / query - response — string — assistant completion - source — string — upstream dataset identifier (e.g., OpenThoughts 3) - messages — list — chat-formatted multi-turn conversation (where applicable) - language — string — ISO language code - +additional metadata columns per source subset (Schema may vary slightly across subset shards; consult HF dataset viewer for exact fields.) ## Sources - allenai/Dolci-Instruct-SFT on HuggingFace — https://huggingface.co/datasets/allenai/Dolci-Instruct-SFT — license: ODC-BY - Upstream component OpenThoughts 3 (Apache 2.0) — 941,166 prompts, extended to 32K context, code prompts downsampled 16x - Additional upstream mixtures documented on the HF dataset page ## Methodology AI2 assembled the mixture by combining multiple existing instruction datasets with annotations from crowdsourced, expert-generated, and machine-generated sources. Specific preprocessing includes extending OpenThoughts 3 prompts to a 32K context length and downsampling code-heavy prompts by 16x to balance the distribution. The final mixture was used directly to train the Olmo 3 7B Instruct SFT checkpoint. See arxiv:2512.13961 for full methodology. ## Known gaps & limitations Language coverage is broad (70+ languages) but heavily skewed toward English and high-resource languages; low-resource language samples may be sparse or machine-translated. The mixture includes machine-generated annotations which may carry model biases from their generating systems. It has not been deduplicated against common evaluation suites — leakage risk exists for benchmarks overlapping with OpenThoughts or other upstream sources. ODC-BY requires attribution; AI2's Responsible Use Guidelines apply. ## Intended use & out-of-scope - IS for: SFT fine-tuning of instruction-following LLMs, reproducing Olmo 3 Instruct training, research on multilingual instruction tuning, ablation studies on prompt mixtures. - NOT for: training models for benchmark evaluation without dedup checks (leakage risk); commercial use without complying with ODC-BY attribution and AI2 Responsible Use terms; assumed-safe deployment without additional safety alignment. _Federated dataset: 15 parquet shards, 2.85 GB total. Queries and downloads stream through the DataBazaar API._ _PII signals: cc_shape×4 (Luhn-valid: 0) present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ ## Temporal validity This dataset includes column(s) keyed on recycled identifiers — the same value can refer to different entities at different times: - **domain name** (reissued by registrars (drop-catching)) — domains are recycled; use WHOIS history to filter enrichment by the current registration interval. Original supplier listing: Dolci Instruct SFT Mixture (Olmo 3 Training Data) 2.15M-sample multilingual instruction SFT mixture used to train AI2's Olmo 3 7B Instruct. Parquet format, ODC-BY licensed, covers 70+ languages and includes OpenThoughts 3 and other curated prompt sets.