OpenCodeInstruct Code LLM Tuning Dataset

## Overview OpenCodeInstruct is NVIDIA's large-scale instruction tuning dataset for code LLMs, comprising approximately 5 million synthetic instruction-response samples designed for supervised fine-tuning (SFT). The dataset is distributed in Parquet format, contains English-language coding instructions across diverse programming tasks, and was released in April 2025. ## Schema - instruction — string — the coding task or question prompt - response — string — model-generated answer/solution code - domain — string — task category or programming domain - language — string — programming language of the sample - difficulty — string — estimated difficulty tier - source — string — generation pipeline source/seed origin - (additional metadata columns may be present; see HF dataset card) ## Sources - nvidia/OpenCodeInstruct on Hugging Face — https://huggingface.co/datasets/nvidia/OpenCodeInstruct — license: CC-BY-4.0 - Technical report: arXiv:2504.04030 ## Methodology Per NVIDIA's technical report, OpenCodeInstruct was constructed via a synthetic data generation pipeline that produces diverse coding instructions and corresponding responses. The pipeline emphasizes diversity across programming domains, languages, and difficulty levels, with quality filtering applied. Full methodology and the SFT pipeline are documented in the linked GitHub repo and arXiv paper. ## Known gaps & limitations As synthetic data generated by LLMs, samples may contain hallucinated APIs, subtly incorrect solutions, or stylistic biases of the generator model. English-only — no multilingual instruction coverage. Not deduplicated against public coding benchmarks (HumanEval, MBPP, LiveCodeBench, etc.), so buyers training evaluation models should run their own decontamination. NVIDIA does not publish per-sample provenance for all seed prompts; buyers should validate empirically for their use case. ## Intended use & out-of-scope - IS for: supervised fine-tuning of code-generation LLMs, instruction tuning research, building coding assistants, distillation studies. - NOT for: benchmark training without decontamination (leakage risk against common code eval suites); not a substitute for human-curated code review data; not validated for safety-critical code generation. _Federated dataset: 7 parquet shards, 1.66 GB total. Queries and downloads stream through the DataBazaar API._ _PII signals: email×45 present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ ## Temporal validity This dataset includes column(s) keyed on recycled identifiers — the same value can refer to different entities at different times: - **domain name** (reissued by registrars (drop-catching)) — domains are recycled; use WHOIS history to filter enrichment by the current registration interval. Original supplier listing: OpenCodeInstruct: 5M Instruction Tuning Samples for Code LLMs (NVIDIA) NVIDIA's 5M-sample open-access instruction tuning dataset for supervised fine-tuning of code LLMs. Synthetic, diverse coding instructions and responses in English, CC-BY-4.0 licensed.