MINT-1T ArXiv Multimodal Documents

## Overview MINT-1T-ArXiv is the ArXiv subset of MINT-1T, a large-scale open-source multimodal interleaved dataset released by the University of Washington's ml-foundations group. It contains interleaved sequences of text and images extracted from ArXiv scientific papers, packaged as WebDataset shards. The full MINT-1T corpus contains ~1 trillion text tokens and 3.4 billion images across HTML, PDF, and ArXiv sources; this listing covers the ArXiv slice (1M–10M documents). Modalities: image + text. Language: English. ## Schema WebDataset tar shards with per-sample records. Key fields: - `__key__` — string — unique sample identifier (typically the ArXiv paper ID) - `json` — JSON — interleaved document structure with ordered text segments and image references - `text` — string — extracted paper text in reading order - `images` — list — embedded or referenced figure images (PNG/JPEG) - `metadata` — JSON — ArXiv source metadata (paper id, etc.) Exact field names follow the MINT-1T WebDataset convention; consult the dataset card for the canonical schema. ## Sources - HuggingFace: https://huggingface.co/datasets/mlfoundations/MINT-1T-ArXiv — License: CC-BY-4.0 - Paper: MINT-1T (arXiv:2406.11271) - Upstream content: ArXiv.org papers (ArXiv's own licensing varies per paper; MINT-1T aggregates and redistributes under CC-BY-4.0 per the dataset card) ## Methodology The ml-foundations team extracted text and figures from ArXiv LaTeX/PDF sources and interleaved them into multimodal document sequences preserving the in-document order of prose and figures. This produces training samples where images appear in context with the surrounding paragraphs, suitable for interleaved multimodal pretraining (à la Flamingo / IDEFICS / OBELICS-style training). Documents are sharded into WebDataset tars for streaming-friendly large-scale training. ## Known gaps & limitations - English-only; non-English ArXiv papers are not represented. - ArXiv coverage skews toward STEM (physics, CS, math); humanities and other disciplines are underrepresented. - Figure extraction from PDFs/LaTeX is imperfect — some figures may be missing, mis-ordered, or rasterized at variable quality. - Underlying ArXiv papers carry per-paper licenses (often arXiv's non-exclusive license, sometimes CC variants); the CC-BY-4.0 designation applies to MINT-1T's compiled form — downstream users should verify if they need to reuse individual figures outside training contexts. - Not deduplicated against common multimodal eval suites — leakage risk for benchmarks built on ArXiv content. ## Intended use & out-of-scope - IS for: large-scale multimodal interleaved pretraining, vision-language model training, scientific document understanding research, RAG over scientific literature. - NOT for: redistribution of individual ArXiv figures outside fair-use/training contexts without checking per-paper licenses; benchmark training where ArXiv leakage matters. _Federated dataset: 10 parquet shards, 3.95 GB total. Queries and downloads stream through the DataBazaar API._ _PII signals: email×10 present in the sample. Common in public datasets (papers, logs) but worth knowing before joining with private data._ Original supplier listing: MINT-1T ArXiv — Multimodal Interleaved ArXiv Papers (Text + Images) ArXiv subset of MINT-1T: multimodal interleaved text-and-image documents extracted from ArXiv papers, designed for multimodal pretraining at scale. CC-BY-4.0.