To search datasets programmatically: GET https://api.databazaar.io/datasets?query=your-search
Full API docs: https://api.databazaar.io/llms.txt
Agent discovery: https://databazaar.io/.well-known/agent.json
Browse Data
73–94 of 94UltraChat 200k (Zephyr SFT Dataset)
Filtered 200k-dialogue subset of UltraChat used to train Zephyr-7B-β. High-quality multi-turn ChatGPT-generated conversations for supervised fine-tuning of chat models. MIT licensed, parquet format.
NuminaMath-TIR: Tool-Integrated Reasoning Math Problems
~70k math problems with tool-integrated reasoning (TIR) traces generated via GPT-4, derived from NuminaMath-CoT. Apache-2.0 licensed, ideal for training math-reasoning agents with code execution.
Full HH-RLHF (Prompt/Chosen/Rejected Format)
Anthropic's Helpful & Harmless RLHF dataset reformatted into prompt/chosen/rejected triples for preference modeling and DPO/RLHF training.
Wikimedia Wikipedia (All Languages, Cleaned)
Cleaned full-text Wikipedia articles across 300+ language subsets, built from official Wikimedia dumps. Parquet format, one row per article. Foundational corpus for LLM pretraining, RAG, and multilingual NLP.
SWE-bench: Real-World GitHub Issue Resolution Benchmark
2,294 Issue-PR pairs from 12 popular Python repos for evaluating LLM/agent ability to resolve real GitHub issues. Verified via post-PR unit tests. Canonical agent coding benchmark.
C4: Colossal Clean Crawled Corpus (en + multilingual mC4)
Cleaned Common Crawl web text corpus from AllenAI/Google. 305GB English + 9.7TB multilingual (108 languages). Foundational pretraining dataset behind T5 and many open LLMs. ODC-BY licensed.
FinePDFs — 3T Tokens from 475M PDFs in 1,733 Languages
Largest public PDF-sourced corpus: ~3 trillion tokens, 475M documents, 1,733 languages. Parquet format, ODC-By licensed. Built by HuggingFaceFW for pretraining and multilingual research.
FineWeb-Edu: 1.3T Tokens of Educational Web Content
1.3 trillion tokens of high-quality educational web pages filtered from FineWeb using a Llama3-70B-trained classifier. Parquet format, ODC-By licensed, ideal for LLM pretraining and RAG.
Common Corpus — 2.27T Tokens of Permissibly-Licensed Text
Largest open and permissibly-licensed text dataset: 2.27 trillion tokens across books, newspapers, scientific articles, legal docs, code, and more in 13+ languages. Built by PleIAs for LLM pretraining.
MMLU — Massive Multitask Language Understanding Benchmark
57-subject multiple-choice benchmark (humanities, STEM, social sciences, professional) for evaluating LLM knowledge and reasoning. ~116K questions across dev/val/test splits. MIT licensed.
FineWeb2: Multilingual Web Pretraining Corpus (1000+ Languages)
Second iteration of HuggingFace's FineWeb dataset, covering 1000+ languages with high-quality filtered web text for LLM pretraining. ODC-By 1.0 licensed, validated through hundreds of ablation experiments.
Wikipedia 2024-06 Embeddings (BGE-M3, Multilingual)
Wikipedia paragraph embeddings from the June 2024 dump across 11 languages, generated with the multilingual BGE-M3 model. Ideal for multilingual RAG, semantic search, and retrieval evals.
SWE-bench Pro
Enterprise-level benchmark dataset from Scale AI for evaluating AI agents on long-horizon software engineering tasks. Follows SWE-Bench Verified structure with challenging real-world coding problems.
SQuAD 2.0 - Stanford Question Answering Dataset
Reading comprehension benchmark with 150K+ questions on Wikipedia articles, including 50K unanswerable questions. Standard for extractive QA model training and evaluation.
MegaMath: 300B+ Token Open Math Pretraining Corpus
Largest open math-focused pretraining dataset (300B+ tokens) from LLM360, curated from Common Crawl, code, and synthetic sources for training math-capable LLMs.
Wikipedia 2023-11 Multilingual Embeddings (Cohere Embed V3, 300+ Languages)
~250M Wikipedia paragraph embeddings across 300+ languages, generated with Cohere Embed V3. Ideal for multilingual semantic search and RAG.
NuminaMath 1.5 — 900K Competition Math Problems with Chain-of-Thought Solutions
~900K competition-level math problems with Chain-of-Thought solutions, sourced from Chinese high school exercises through international olympiads. Apache 2.0, parquet format, ideal for math reasoning fine-tuning and RAG.
Anthropic HH-RLHF: Helpful & Harmless Human Preference Data
Anthropic's human preference dataset for training helpful and harmless assistants via RLHF. ~170K chosen/rejected response pairs covering helpfulness and red-teaming harmlessness data. MIT licensed.
SQuAD 1.1 — Stanford Question Answering Dataset
100K+ crowdsourced question-answer pairs on Wikipedia passages. Canonical extractive QA benchmark for reading comprehension, RAG eval, and fine-tuning.
Public Domain Books Catalog — 75,000+ Literary Works (1971–2025)
Cross-national catalog of 75,545 public domain literary works from Project Gutenberg, enriched with genre classification, literary era mapping, and Library of Congress subject area categorization. Covers works in 58+ languages from ancient texts to early 20th-century literature. **Sources:** - Project Gutenberg digital library catalog (primary metadata: titles, authors, dates, subjects, Library of Congress Classification) - Library of Congress Classification scheme (subject area mapping) - Literary period taxonomy (era classification from Medieval through Contemporary) - Custom NLP-derived genre classification across 20+ categories **Schema (23 columns):** - `gutenberg_id` — Unique Project Gutenberg text identifier - `title` — Full title of the work - `author` — Primary author name (normalized to "First Last" format) - `author_birth_year` / `author_death_year` — Author life dates - `num_authors` — Number of credited authors - `language_code` — ISO language code - `language` — Full language name - `issued_date` — Date digitized/added to Project Gutenberg - `primary_subject` — Primary subject heading - `subject_count` — Total number of subject headings - `locc_classification` — Library of Congress Classification code(s) - `locc_area` — Mapped LoCC broad subject area - `genre` — Derived genre (Fiction, Poetry, History, Science Fiction, Mystery, etc.) - `literary_era` — Estimated literary period (Medieval, Renaissance, Romantic, Victorian, Modern, Contemporary) - `bookshelf` — Project Gutenberg bookshelf category - `source` — Data source identifier - `url` — Direct link to the work - `license` — License type (all Public Domain) - `title_word_count` — Number of words in title - `has_author` — Whether author is known (1/0) - `is_english` — English language flag (1/0) - `has_classification` — Has LoCC classification (1/0) **Coverage:** 75,545 unique works across 58+ languages. 60K+ English works plus significant French (4K), Finnish (3.5K), German (2.3K), and 50+ other language collections. Literary eras span from Ancient/Medieval through Contemporary. **Use cases:** Literary analysis, NLP training data catalogs, bibliometric research, digital humanities, author network analysis, genre classification benchmarking, language diversity studies, cultural heritage research.
English Vocabulary & Linguistic Properties — 15,000 Words with Frequency, POS, Morphology & Phonetics
A comprehensive dataset of 15,000 English words enriched with 25 linguistic properties, ideal for NLP research, computational linguistics, language learning applications, and word game development. **Data Sources:** Datamuse API (word frequency, definitions, syllable counts, parts of speech) combined with computed morphological and phonetic properties. **Key Features:** - 15,000 words ranked by corpus frequency (most common to rare) - Part-of-speech tags (noun, verb, adjective, adverb) - Syllable counts and consonant-vowel patterns - Morphological analysis (common prefixes and suffixes) - Scrabble scores and complexity tiers (basic/intermediate/advanced) - Primary definitions from Wiktionary - Frequency data from large-scale English corpora **Columns (25):** word, length, syllable_count, primary_pos, all_pos, num_definitions, primary_definition, corpus_frequency, frequency_rank, vowel_count, consonant_count, vowel_ratio, unique_letter_count, unique_letter_ratio, starts_with, ends_with, cv_pattern, detected_prefix, detected_suffix, has_double_letters, is_palindrome, is_monosyllabic, is_polysyllabic, scrabble_score, complexity_tier **Use Cases:** - NLP model training and evaluation - Word difficulty scoring for education apps - Vocabulary analysis and readability tools - Word game engines (Scrabble, crosswords, Wordle-style) - Linguistic research on English morphology
100 Dad Jokes
A curated dataset of 100 clean dad jokes in CSV format. Each row includes a setup and punchline. Includes a 10-row preview sample for quick evaluation. Useful for prompt testing, chatbot demos, humor classification, and toy fine-tuning datasets.