MATH Competition Mathematics Problems Benchmark

## Overview The MATH (Mathematics Aptitude Test of Heuristics) dataset contains roughly 12,500 challenging competition mathematics problems sourced from contests including AMC 10, AMC 12, AIME, and similar olympiad-style events. Each problem is paired with a full step-by-step natural-language solution, making it a canonical resource for training and evaluating mathematical reasoning, chain-of-thought generation, and answer-derivation behavior in LLMs. Format is parquet; modality is text (English only). ## Schema - `problem` — string — the natural-language problem statement, often with LaTeX math - `level` — string — difficulty label (e.g., "Level 1" through "Level 5") - `type` — string — subject category (Algebra, Geometry, Number Theory, Counting & Probability, Intermediate Algebra, Prealgebra, Precalculus) - `solution` — string — full step-by-step worked solution including final boxed answer ## Sources - Original dataset: Hendrycks et al., "Measuring Mathematical Problem Solving With the MATH Dataset" (arXiv:2103.03874) - HuggingFace mirror: https://huggingface.co/datasets/qwedsacf/competition_math — license: MIT ## Methodology Problems were collected from publicly available U.S. high-school and undergraduate mathematics competitions (AMC, AIME, and similar). Human experts transcribed problems and authored or curated step-by-step solutions, with final answers enclosed in `\boxed{}` for programmatic extraction. The authors assigned difficulty levels (1–5) and subject categories. No automated normalization beyond LaTeX-consistent formatting is documented. ## Known gaps & limitations English-only and U.S.-competition-centric, so problem style and notation reflect that tradition. The dataset is widely used as a benchmark and is known to appear in pretraining corpora of many frontier LLMs, creating significant contamination/leakage risk when used for evaluation. Solutions are human-written but not formally verified; occasional errors have been reported by the community. Difficulty labels are author-assigned and approximate. Dataset is static (2021 vintage) and contains no problems from contests after that period. ## Intended use & out-of-scope - IS for: training and evaluating mathematical reasoning, chain-of-thought fine-tuning, supervised fine-tuning on derivations, RAG over solved math problems, and research on step-by-step problem solving. - NOT for: clean held-out evaluation of frontier LLMs without contamination checks (likely leaked into pretraining); not a substitute for formal proof datasets; not multilingual. Original supplier listing: MATH: Competition Mathematics Problems with Step-by-Step Solutions 12,500 competition math problems (AMC 10/12, AIME, etc.) with full worked solutions. Standard benchmark for math reasoning, chain-of-thought training, and LLM evaluation. MIT licensed, parquet format.