UGMathBench Undergraduate Math Reasoning Benchmark

## Overview UGMathBench is a benchmark for evaluating undergraduate-level mathematical reasoning in large language models. It contains 5,062 problems spanning 16 subjects and 111 topics, with 10 distinct answer types. Each problem ships with three randomized versions to test robustness against memorization. Data is distributed as JSON; the total set sits in the 1K–10K row size bucket per problem, with version expansions on top. ## Schema - problem_id — string — unique problem identifier - subject — string — one of 16 undergraduate math subjects - topic — string — finer-grained topic label (111 values) - problem — string — natural-language problem statement (LaTeX where relevant) - answer_type — string — one of 10 answer formats (numeric, expression, multiple choice, etc.) - answer — string — ground-truth answer - version — integer/string — which of the 3 randomized variants this row represents - solution — string — reference solution where provided ## Sources - UGMathBench on Hugging Face: https://huggingface.co/datasets/UGMathBench/ugmathbench — license: GPL-3.0 - Paper: https://huggingface.co/papers/2501.13766 (arXiv:2501.13766) ## Methodology The authors curated problems from undergraduate mathematics course materials covering 16 subjects and 111 topics, then generated three randomized versions of each problem (varying numerical values or surface form) to mitigate test-set memorization and enable robustness evaluation. Answers are typed into 10 categories to support automated grading. See the paper for full curation and randomization protocol. ## Known gaps & limitations English-only. Coverage is undergraduate-level and may not represent graduate research mathematics or competition math. The three randomized versions are derived from the same source problem, so they are not statistically independent samples. The GPL-3.0 license is share-alike and copyleft — downstream derivative datasets may inherit GPL obligations. Source does not document demographic or institutional sampling biases; buyers should validate empirically for their use case. ## Intended use & out-of-scope - For: evaluating and benchmarking LLM mathematical reasoning, robustness testing via problem variants, and analysis of subject/topic-level performance. - Not for: fine-tuning models you then plan to evaluate on UGMathBench itself (leakage risk); also not a substitute for competition-math or research-math benchmarks. _Federated dataset: 16 parquet shards, 3.4 MB total. Queries and downloads stream through the DataBazaar API._ _PII signals: cc_shape×3 (Luhn-valid: 0) present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ Original supplier listing: UGMathBench: Undergraduate Math Reasoning Benchmark 5,062 undergraduate-level math problems across 16 subjects and 111 topics, with 10 answer types and 3 randomized versions each. Designed for evaluating LLM mathematical reasoning.