TextCaps Image Captioning Benchmark

## Overview TextCaps is an image captioning dataset that specifically requires models to read and reason about text appearing in images. This is the lmms-lab formatted version, prepared for use in the lmms-eval evaluation pipeline for large multimodality models (LMMs). The dataset contains roughly 28K images with multiple reference captions each, distributed in parquet format with image and text modalities. Size category is 10K-100K rows. ## Schema - image — image (bytes) — the source image containing text to be read - image_id — string — unique identifier for the image - caption_str — string — reference caption(s) describing the image (including OCR'd text) - reference_strs — list[string] — multiple human reference captions for evaluation - ocr_tokens — list[string] — OCR tokens extracted from the image (where provided) - question_id / set_name — string — evaluation split metadata ## Sources - HuggingFace: https://huggingface.co/datasets/lmms-lab/TextCaps - Original TextCaps paper: Sidorov et al., ECCV 2020 — "TextCaps: a Dataset for Image Captioning with Reading Comprehension" - License: per source (TextCaps original is CC-BY 4.0; images sourced from Open Images) ## Methodology The original TextCaps was constructed by selecting images from the Open Images dataset that contain text, then crowdsourcing multiple captions per image where annotators were instructed to describe the image including any text present. lmms-lab reformatted the dataset into parquet with embedded images and standardized fields for plug-and-play evaluation in their lmms-eval framework. ## Known gaps & limitations English captions only; OCR difficulty varies and some text in images may be unreadable. Images are sourced from Open Images so biases of that collection (Western, web-scraped) carry over. The lmms-lab reformatting is for evaluation use — the train split formatting may differ from the original release. Source does not exhaustively document reformatting deltas; buyers should validate against the canonical TextCaps release if exact parity matters. ## Intended use & out-of-scope - IS for: evaluating multimodal/vision-language models on OCR-aware image captioning, benchmarking LMMs via lmms-eval, RAG over image+text content. - NOT for: training production captioning systems without deduplication against the eval set; this is a public benchmark with high contamination risk for any model trained on web data. _Federated dataset: 17 parquet shards, 7.51 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: TextCaps (lmms-eval formatted) Image captioning benchmark requiring OCR/text reading in images. Formatted by lmms-lab for one-click multimodal model evaluation. ~28K images with captions.