Nemotron VLM Dataset v2

## Overview Nemotron-VLM-Dataset-v2 is NVIDIA's second-generation vision-language training corpus, expanding on the 3M-sample V1 release to approximately 9M high-quality multimodal samples. It targets visual question answering, image-to-text, video-to-text, and document understanding tasks, and was used in training NVIDIA's Nemotron VLM models. Distributed as JSON files with image/video references; size category 1M–10M rows. ## Schema The dataset is organized into multiple sub-corpora (e.g., nights_cot, fintabnet, document-understanding splits). Typical fields include: - `id` — string — unique sample identifier - `image` / `video` — string — path or reference to the visual asset - `conversations` — list — multi-turn user/assistant messages (instruction-tuning format) - `question` — string — user prompt or VQA query - `answer` — string — target response - `think` — string — optional chain-of-thought reasoning trace (filtered in v2 patch) - `source` — string — sub-dataset origin - `task_type` — string — VQA, captioning, document QA, etc. - `language` — string — language code (multilingual coverage) - +N more columns depending on sub-dataset ## Sources - nvidia/Nemotron-VLM-Dataset-v2 on Hugging Face — https://huggingface.co/datasets/nvidia/Nemotron-VLM-Dataset-v2 — license: CC-BY-4.0 - Companion paper: arXiv:2511.03929 ## Methodology NVIDIA curated and generated samples from a mix of public vision-language datasets and synthetic data pipelines, focused on document understanding, multilingual visual reasoning, and chain-of-thought VQA. The v2 patch (2025-11-05) fixed the nights_cot subset, filtered broken `<think>` entries, updated fintabnet instructions, and refreshed indexes. Samples are formatted as instruction-tuning conversations suitable for VLM SFT. ## Known gaps & limitations Source flags fixes to broken chain-of-thought entries in the initial release, indicating some residual data-quality risk. Multilingual coverage is uneven across sub-datasets, and NVIDIA does not publish a full bias/coverage audit. Some sub-corpora reference upstream images/videos that must be downloaded separately. Buyers should validate empirically for their target task and check for overlap with public eval benchmarks. ## Intended use & out-of-scope - IS for: SFT / instruction-tuning of vision-language models, document-understanding fine-tuning, multilingual VQA training, chain-of-thought VLM research. - NOT for: drop-in benchmark evaluation without leakage checks — overlap with common VQA/document eval suites is not documented; not a standalone image corpus (many entries reference external assets). _Federated dataset: 57 parquet shards, 4.36 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: Nemotron VLM Dataset v2 (NVIDIA) NVIDIA's large-scale vision-language training dataset (~9M samples) for VQA, image-text-to-text, video-text-to-text, and document understanding. CC-BY-4.0.