Salad-Data LLM Safety Evaluation Benchmark

## Overview Salad-Data is a safety evaluation dataset for large language models containing 21,318 harmful or sensitive prompts curated for red-teaming, jailbreak detection, and safety alignment research. The base set aggregates prompts from multiple established safety benchmarks plus GPT-3.5-generated self-instructed examples. Format is JSON; modalities are tabular/text. Released alongside the SALAD-Bench paper (arXiv:2402.05044). ## Schema - `question` — string — the harmful/sensitive prompt to test the model with - `qid` — string/int — unique question identifier - `source` — string — origin source (HH-harmless, AdvBench, ToxicChat, GPTFuzzer, self-instructed, etc.) - `1-category` — string — top-level harm taxonomy label - `2-category` — string — mid-level harm category - `3-category` — string — fine-grained harm category - Multiple subsets available via `name=` parameter (e.g., `base_set`, attack-enhanced sets, MCQ sets) ## Sources - HuggingFace: https://huggingface.co/datasets/OpenSafetyLab/Salad-Data — Apache-2.0 - Underlying sources: Anthropic HH-RLHF (harmless + red-team splits), AdvBench, Do-Not-Answer, ToxicChat, GPTFuzzer, Do Anything Now (DAN), Multilingual safety prompts, plus GPT-3.5 self-instructed generation - Paper: SALAD-Bench (arXiv:2402.05044) ## Methodology The authors aggregated harmful prompts from eight existing open-source safety benchmarks (15,885 prompts) and augmented them with 15,433 self-instructed prompts generated by a fine-tuned GPT-3.5 to broaden coverage across a hierarchical 3-level harm taxonomy (6 domains → 16 tasks → 66 categories). Prompts were deduplicated and categorized; the paper also introduces attack-enhanced and defense-enhanced variants generated by applying jailbreak templates to the base set. ## Known gaps & limitations Primarily English with a small multilingual subset (230 prompts); harm taxonomy reflects authors' choices and may not align with other safety frameworks. Self-instructed prompts inherit GPT-3.5 biases. Dataset is from early 2024 — newer jailbreak techniques (e.g., recent multi-turn or vision-based attacks) are not represented. Significant overlap with widely-used benchmarks (AdvBench, HH-RLHF) means leakage risk if used to train models that will be evaluated on those benchmarks. ## Intended use & out-of-scope - **For**: LLM safety evaluation, red-team testing, jailbreak detection research, safety classifier training, RLHF data augmentation - **Not for**: training general-purpose chatbots without safety filtering; benchmarking against AdvBench/HH-RLHF (leakage); production content moderation without additional validation _Federated dataset: 4 parquet shards, 5.8 MB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: Salad-Data: LLM Safety & Jailbreak Evaluation Benchmark 21K+ safety questions for LLM red-teaming and jailbreak evaluation, aggregated from HH-RLHF, AdvBench, ToxicChat, GPTFuzzer, and GPT-3.5 self-instructed prompts. Apache-2.0 licensed.