HelpSteer3 Multilingual Preference Dataset

## Overview HelpSteer3 is NVIDIA's third-generation open-source human preference dataset for aligning LLMs to be more helpful. Released under CC-BY-4.0, it contains 100K–1M samples in JSON format spanning 15 languages (English, Chinese, Korean, French, Spanish, Russian, Japanese, German, Italian, Portuguese, Polish, Indonesian, Dutch, Vietnamese). It includes preference pairs (HelpSteer3-Preference), free-text feedback (HelpSteer3-Feedback), and edit data. Last updated November 2025. ## Schema - prompt — string — the user request/instruction - response1 / response2 — string — candidate model responses being compared - overall_preference — int — preference label (typically -3 to +3 scale indicating which response is better and by how much) - individual_preference — list — per-annotator preference judgments with rationales - domain — string — task domain (e.g., code, general, multilingual) - language — string — language code of the sample - feedback — string — free-text annotator critique (Feedback split) - context — list — multi-turn conversation context where applicable - +several more columns across the Preference/Feedback/Edit subsets ## Sources - nvidia/HelpSteer3 on Hugging Face — https://huggingface.co/datasets/nvidia/HelpSteer3 — License: CC-BY-4.0 - Associated papers: arXiv:2505.11475 (HelpSteer3), arXiv:2503.04378, arXiv:2509.21319, arXiv:2410.16184 ## Methodology Prompts were sourced from real-world LLM usage and synthetic generation, then paired with responses from multiple models. Human annotators provided preference ratings, written feedback, and in some cases response edits. NVIDIA expanded coverage beyond English (unlike HelpSteer2) to include 14 additional languages and added domain coverage for code and multi-turn dialogue. Annotation followed structured guidelines for helpfulness, correctness, coherence, complexity, and verbosity. ## Known gaps & limitations Multilingual coverage is uneven — English dominates, with smaller samples per non-English language. Preference annotations reflect annotator subjectivity and may carry cultural/demographic biases from NVIDIA's annotator pool. The dataset is optimized for training reward models; direct SFT use may require filtering. Source does not exhaustively document per-language sample counts or annotator demographics; buyers should validate empirically for their alignment use case. ## Intended use & out-of-scope - IS for: training reward models (Bradley-Terry and Generative RMs), RLHF/DPO fine-tuning, preference learning research, multilingual alignment, and RM benchmark development. - NOT for: deduplication-free benchmark training (overlap risk with RM-Bench/JudgeBench evaluation prompts has not been audited), safety-critical alignment without additional red-teaming, or as a sole source of ground-truth correctness labels. _Federated dataset: 10 parquet shards, 498.1 MB total. Queries and downloads stream through the DataBazaar API._ ## Temporal validity This dataset includes column(s) keyed on recycled identifiers — the same value can refer to different entities at different times: - **domain name** (reissued by registrars (drop-catching)) — domains are recycled; use WHOIS history to filter enrichment by the current registration interval. Original supplier listing: HelpSteer3 — NVIDIA Human Preference & Feedback Dataset for RLHF/Reward Modeling NVIDIA's open-source multilingual preference dataset for training reward models and aligning LLMs. CC-BY-4.0, 100K+ samples across 15 languages, used to train SOTA reward models on RM-Bench (85.5%) and JudgeBench (78.6%).