WildChat-4.8M — Filtered Conversation Subset

## Overview WildChat-4.8M is a large-scale corpus of real conversations between human users and ChatGPT (GPT-3.5 and GPT-4), collected by the Allen Institute for AI by offering free ChatGPT access in exchange for consent to release chat logs. This release contains 3,199,860 conversations (the non-toxic subset, with toxic turns filtered via the OpenAI Moderations API). Data is distributed as parquet files containing multi-turn conversations with metadata. Useful for instruction-tuning, dialogue modeling, user-intent analysis, and studying real-world LLM usage patterns. ## Schema - `conversation_id` — string — unique conversation identifier - `model` — string — underlying model (e.g. gpt-3.5-turbo, gpt-4) - `timestamp` — datetime — conversation time - `conversation` — list[dict] — ordered turns with `role` (user/assistant) and `content` - `turn` — int — number of turns in the conversation - `language` — string — detected language of the conversation - `openai_moderation` — list[dict] — per-turn moderation scores from OpenAI Moderations API - `detoxify_moderation` — list[dict] — per-turn Detoxify scores - `toxic` — bool — overall toxicity flag (false for this subset) - `redacted` — bool — whether PII redaction was applied - `state`, `country`, `hashed_ip` — coarse geo metadata derived from IP ## Sources - Hugging Face: https://huggingface.co/datasets/allenai/WildChat-4.8M — license: ODC-By - Paper: WildChat (https://arxiv.org/abs/2405.01470) - Interactive explorer: https://wildvisualizer.com ## Methodology AI2 deployed free ChatGPT and GPT-4 chatbots online; users consented to release their interactions in exchange for free access. Conversations were logged server-side along with model, timestamp, hashed IP, and coarse geolocation. Each turn was scored by the OpenAI Moderations API and Detoxify. PII was redacted from user inputs. This 4.8M release expands earlier WildChat-1M; the published subset here contains 3.2M conversations after filtering out turns flagged as toxic by either moderation pipeline. ## Known gaps & limitations - Geographic and demographic skew toward users who discovered the free chatbot service; not a representative sample of global ChatGPT usage. - Heavy English skew, though many languages appear in the long tail. - Moderation filtering relies on OpenAI Moderations + Detoxify, both of which have known false-positive and false-negative patterns; "non-toxic" should not be taken as safe-for-all-audiences. - Conversations reflect ChatGPT outputs from a specific timeframe; model behavior and capabilities have shifted since. - PII redaction is automated and imperfect — buyers handling user content should apply additional filtering. ## Intended use & out-of-scope - IS for: instruction-tuning data, real-world prompt distribution analysis, RAG over user-intent corpora, dialogue eval construction, studying user behavior with LLMs. - NOT for: training models intended to impersonate specific users; production deployment without additional PII/safety review; benchmarking against eval suites that may overlap (no deduplication against common eval sets is documented — leakage risk). _Federated dataset: 86 parquet shards, 14.23 GB total. Queries and downloads stream through the DataBazaar API._ _PII signals: us_phone×13, credit_card_candidate×6 (0.3≤score<0.7) present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ Original supplier listing: WildChat-4.8M: Real Human-ChatGPT Conversations (Non-Toxic Subset) 3.2M real user↔ChatGPT conversations collected in the wild by AI2. Non-toxic subset, parquet format, ODC-By licensed. Widely used for instruction tuning, RAG, and chat-model evaluation.