Nemotron Content Safety Dataset V2

## Overview The Nemotron Content Safety Dataset V2 (formerly Aegis AI Content Safety Dataset 2.0) contains 33,416 annotated interactions between humans and LLMs labeled for content safety. The dataset is split into 30,007 training samples, 1,445 validation samples, and 1,964 test samples. Content is in English, JSON-format, and covers a broad taxonomy of harm categories used by NVIDIA's NeMo Guard safety models. ## Schema - `prompt` — string — the human user input - `response` — string — the LLM response (where applicable) - `prompt_label` — string — safe / unsafe label for the prompt - `response_label` — string — safe / unsafe label for the response - `violated_categories` — string — comma-separated harm categories triggered - `prompt_label_source` — string — annotation source/provenance - `response_label_source` — string — annotation source/provenance - Plus additional metadata columns describing annotator agreement and split assignment ## Sources - nvidia/Aegis-AI-Content-Safety-Dataset-2.0 on HuggingFace — https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-2.0 — License: CC-BY-4.0 - Built on top of Anthropic HH-RLHF harmlessness preference data as a seed source ## Methodology NVIDIA curated this dataset by sourcing human prompts from the HuggingFace HH-RLHF harmlessness preference data and generating responses with multiple LLMs. Prompts and responses were then annotated by trained human annotators against NVIDIA's content safety risk taxonomy (12+ categories including hate, harassment, sexual content, violence, self-harm, criminal planning, etc.). V2 extends V1 with additional samples, refined taxonomy, and improved annotation agreement. The dataset is pre-split into train/validation/test partitions. ## Known gaps & limitations English-only — no multilingual coverage. Annotation reflects NVIDIA's specific safety taxonomy which may not align with other policy frameworks (OpenAI, Anthropic, Meta Llama Guard). Some prompts derive from HH-RLHF, which has its own known annotator biases. Class imbalance exists across harm categories. Source documents annotator agreement metrics but buyers should validate empirically for their specific safety policy. ## Intended use & out-of-scope - IS for: training and evaluating content safety classifiers, fine-tuning guardrail models (NeMo Guard, Llama Guard-style), benchmarking moderation systems, red-team eval construction. - NOT for: deployment as a sole arbiter of safety without human review; direct generation training (it contains unsafe content by design); use in non-English moderation pipelines without translation/validation. _Federated dataset: 3 parquet shards, 10.8 MB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: Nemotron Content Safety Dataset V2 (Aegis 2.0) 33,416 annotated human-LLM interactions for content safety classification across 12+ harm categories. Used for training and evaluating LLM safety guardrails like NeMo Guard.