FineInstructions Nemotron Synthetic Instruction-Answer Pairs

## Overview FineInstructions Nemotron is a large-scale synthetic instruction-tuning corpus containing approximately 1 billion+ instruction-answer pairs (roughly 300B tokens) generated by running the FineInstructions pipeline over documents in the Nemotron-CC pre-training corpus, itself a curated high-quality subset of CommonCrawl. The dataset is distributed as Parquet shards in the `data/` folder, each paired with a `judge-*.json` file containing quality/judge scores. Language is English. ## Schema - `instruction` — string — synthetic user instruction grounded in a source web document - `answer` / `response` — string — synthetic assistant response - `source_document` — string — the originating Nemotron-CC document or reference - `judge_score` — float/int — quality score from a judge model (in companion judge-*.json files) - additional pipeline metadata columns (document id, generation params) — see HF dataset viewer for full schema ## Sources - HuggingFace: https://huggingface.co/datasets/fineinstructions/fineinstructions_nemotron — see dataset card for license slug - Upstream: Nemotron-CC corpus (NVIDIA, derived from CommonCrawl) - Paper: arXiv:2601.22146 (FineInstructions methodology) ## Methodology The creators ran the FineInstructions synthetic-data pipeline over raw pre-training documents from Nemotron-CC — itself a high-quality filtered subset of CommonCrawl curated by NVIDIA. The pipeline generates instruction-response pairs grounded in source documents and applies a judge model to produce per-example quality scores stored in companion JSON files. See the FineInstructions paper for full pipeline details including prompt templates, generator/judge model choices, and filtering thresholds. ## Known gaps & limitations - English-only; no multilingual coverage. - Fully synthetic — instruction and answer quality is bounded by the generator model; factual hallucination risk is non-trivial. - Provenance traces back to CommonCrawl, which may include copyrighted, biased, or low-quality web content despite Nemotron-CC filtering. - Not deduplicated against common evaluation suites — leakage risk if used to train models evaluated on standard benchmarks. - Source does not exhaustively document demographic, topical, or domain coverage gaps; buyers should validate empirically for their use case. ## Intended use & out-of-scope - IS for: large-scale instruction fine-tuning, SFT pretraining mixtures, synthetic-data research, distillation, and agent training corpora. - NOT for: benchmark evaluation training without leakage checks; high-stakes factual applications without verification; non-English use cases. _Federated dataset: 1,909 parquet shards, 1593.32 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: FineInstructions Nemotron — 1B+ Synthetic Instruction-Answer Pairs Approximately 1 billion synthetic instruction-answer pairs (~300B tokens) generated via the FineInstructions pipeline over the Nemotron-CC high-quality CommonCrawl pre-training corpus. Parquet format with per-shard judge scoring files.