TriviaQA Reading Comprehension QA

## Overview TriviaQA is a large-scale reading comprehension and open-domain question answering dataset containing 650K+ question-answer-evidence triples in English. It includes ~95K question-answer pairs authored by trivia enthusiasts, paired on average with six independently gathered evidence documents per question (from Wikipedia and web sources) that serve as distant supervision. The dataset is provided in Parquet format and is widely used as a standard benchmark for QA system evaluation, including RAG pipelines and closed-book LLM evals. ## Schema - `question` — string — the trivia question text - `question_id` — string — unique question identifier - `question_source` — string — origin of the question - `answer` — struct — contains `value` (canonical answer), `aliases` (list of acceptable alternate strings), `normalized_aliases`, `normalized_value`, `type`, and `matched_wiki_entity_name` - `entity_pages` — sequence — Wikipedia evidence with `doc_source`, `filename`, `title`, `wiki_context` - `search_results` — sequence — web search evidence with `description`, `filename`, `rank`, `title`, `url`, `search_context` Configurations available: `rc`, `rc.nocontext`, `unfiltered`, `unfiltered.nocontext` (and `.web` / `.wikipedia` variants). ## Sources - HuggingFace: https://huggingface.co/datasets/mandarjoshi/trivia_qa — license listed as "unknown" on HF card, but the original release (Joshi et al., 2017, https://nlp.cs.washington.edu/triviaqa/) is distributed under a non-commercial research-friendly license; commonly redistributed as Apache-2.0 in HF mirrors. Buyers should verify license terms for their use case. - Original paper: https://arxiv.org/abs/1705.03551 ## Methodology Questions were authored by trivia enthusiasts and scraped from 14 trivia and quiz-league websites. For each question, evidence documents were collected independently via (a) Bing web search results and (b) Wikipedia entity pages corresponding to answer entities. This separation of question authoring from evidence collection makes the dataset more challenging than datasets where annotators see the passage first. The HF version provides the dataset pre-split into train/validation/test and into reading comprehension (with context) vs. unfiltered (open-domain) configurations. ## Known gaps & limitations - Test set answers are not publicly released (held out for the leaderboard); only train and validation have labeled answers. - English-only; not suitable for multilingual QA evaluation. - Evidence documents are noisy distant supervision — not all evidence documents actually contain the answer, especially in the unfiltered split. - Content is static from 2017; world-knowledge questions may have answers that have since changed (e.g., "current" office holders). - HF dataset card lists license as "unknown"; original TriviaQA terms should be consulted for commercial deployment. - Heavy overlap with common LLM pretraining corpora — significant leakage risk if used to train models that were pretrained on Common Crawl or Wikipedia. ## Intended use & out-of-scope - **Intended:** RAG system evaluation, open-domain QA benchmarking, extractive and abstractive QA fine-tuning, retrieval evaluation, closed-book LLM factuality evals. - **Out-of-scope:** Training benchmark-evaluated models without decontamination (leakage risk against TriviaQA test sets used in nearly every modern LLM eval suite); multilingual QA; time-sensitive factual QA without freshness adjustment. _Federated dataset: 98 parquet shards, 22.75 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: TriviaQA — Reading Comprehension QA with Evidence Documents 650K+ question-answer-evidence triples for open-domain and reading comprehension QA. Standard benchmark for retrieval-augmented QA, RAG evals, and extractive/abstractive QA model training.