LLaVA-OneVision-1.5 Instruction Data

## Overview LLaVA-OneVision-1.5-Instruct is a large-scale multimodal instruction-tuning dataset of approximately 22 million image-text instruction examples, assembled and curated by the mvp-lab team during development of the LLaVA-OneVision-1.5 family of Large Multimodal Models (LMMs). The corpus aggregates and normalizes data across visual question answering, image captioning, OCR, document understanding, chart/table reasoning, grounding, and general multimodal instruction following. Modality: image + text. Task: image-text-to-text. Language: primarily English. ## Schema - `image` — image (or image reference) — the visual input associated with the instruction - `conversations` / `messages` — list — multi-turn instruction/response pairs in LLaVA-style chat format (roles: human/gpt or user/assistant) - `id` — string — unique sample identifier - `source` / `dataset` — string — original sub-dataset the sample was sourced from (used for mixture weighting and provenance) - Additional task-specific fields may appear depending on sub-source (bounding boxes, OCR text, etc.) Exact field names follow the LLaVA-OneVision conversation schema; see the HF dataset card for the canonical layout. ## Sources - HuggingFace: https://huggingface.co/datasets/mvp-lab/LLaVA-OneVision-1.5-Instruct-Data — License: Apache-2.0 - Paper: arXiv:2509.23661 (LLaVA-OneVision-1.5) - Underlying sub-datasets: an aggregation of many public multimodal instruction datasets (VQA, captioning, OCR, chart/doc, grounding, etc.) — see the dataset card for the full constituent list and their respective upstream licenses. ## Methodology The authors collected, filtered, reformatted, and re-balanced a large set of public multimodal instruction datasets into a unified LLaVA-style conversational schema, producing ~22M curated samples used for the instruction-tuning stage of LLaVA-OneVision-1.5. Curation steps described by the authors include source selection, format unification, mixture weighting, and quality/duplicate filtering aimed at reducing training cost while preserving downstream performance. ## Known gaps & limitations - Constituent sub-datasets each carry their own licenses and biases; the Apache-2.0 tag applies to the aggregation/curation work — downstream users should verify compatibility for any sub-source they care about. - Predominantly English; multilingual coverage is limited and not systematically documented. - Potential overlap with public multimodal evaluation benchmarks (e.g. MMMU, MMBench, ChartQA, DocVQA, TextVQA) — leakage risk has not been exhaustively audited. - Image quality, safety filtering, and PII handling depend on upstream sources; the dataset card does not fully document residual risks. Buyers should validate empirically. ## Intended use & out-of-scope - Intended: instruction tuning / SFT of vision-language models, multimodal RAG augmentation, ablation studies on instruction mixtures, reproducing or extending LLaVA-OneVision-1.5. - Out of scope: training on benchmarks you also plan to evaluate on without deduplication; safety-critical deployments without additional filtering; non-English-heavy applications without supplementary data. _Federated dataset: 6,695 parquet shards, 3678.02 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: LLaVA-OneVision-1.5 Instruction Data (22M Multimodal SFT) 22M curated instruction-tuning examples (image+text) used to train LLaVA-OneVision-1.5 LMMs. Apache-2.0, covers VQA, captioning, OCR, reasoning, and more.