OpenAssistant Conversations (OASST1) Multilingual

## Overview OpenAssistant Conversations (OASST1) is a human-generated, human-annotated assistant-style conversation corpus containing 161,443 messages organized into over 10,000 fully annotated conversation trees, spanning 35 languages. Each message carries quality ratings (461,292 total annotations) covering attributes like helpfulness, creativity, and toxicity. Released April 2023 as parquet, the dataset is widely used for instruction tuning, reward modeling, and RLHF research. ## Schema - message_id — string — unique message identifier - parent_id — string — parent message in the conversation tree (null for root prompts) - text — string — the message content - role — string — 'prompter' or 'assistant' - lang — string — ISO language code - review_count — int — number of human reviews - review_result — bool — whether message passed review - deleted — bool — moderation flag - rank — int — sibling ranking among assistant replies - labels — struct — quality/safety annotations (quality, toxicity, humor, creativity, etc.) with values and counts - tree_state — string — state of the conversation tree (ready_for_export, etc.) - +several metadata columns (emojis, synthetic, model_name, detoxify scores) ## Sources - OpenAssistant/oasst1 on Hugging Face — https://huggingface.co/datasets/OpenAssistant/oasst1 — Apache-2.0 - Paper: Köpf et al., "OpenAssistant Conversations" (arXiv:2304.07327) ## Methodology Collected via a worldwide crowd-sourcing effort on the open-assistant.io platform. Volunteers wrote prompts, wrote assistant replies, and rated/ranked other contributors' messages along multiple labeled dimensions. Conversation trees were grown by branching: multiple assistant replies per prompt, ranked by reviewers. Messages flagged by reviewers or detoxify thresholds were filtered, and the released split contains only trees in a 'ready_for_export' state. Train/validation splits are provided by the publisher. ## Known gaps & limitations Language distribution is heavily skewed toward English, Spanish, Russian, and German; many of the 35 languages have only a small number of messages. Crowd-sourced contributors are self-selected and not demographically representative. Quality annotations are subjective and reviewer counts per message vary, so per-label confidence is uneven. The dataset captures the state of contributions as of April 2023 and is not updated; OASST2 supersedes it for newer work. Some safety-related content remains in the corpus by design for alignment research — buyers should filter for their use case. ## Intended use & out-of-scope - For: instruction tuning, RLHF/DPO reward model training, multilingual assistant fine-tuning, alignment research, conversation-tree evaluation. - Not for: production safety guarantees without additional filtering; benchmarks where leakage into popular open-source instruct models (Llama-2-Chat derivatives, many community models) is a concern — assume contamination across most current open chat LLMs. _Federated dataset: 2 parquet shards, 39.7 MB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: OpenAssistant Conversations (OASST1) 161,443 human-generated assistant conversation messages across 35 languages with 461,292 quality ratings — a foundational dataset for alignment, RLHF, and instruction-tuning research.