textai4bharat/sangrahaindicmultilingualpretrainingllmsouth-asianparquetcc-byai4bharattext-generationlow-resource-languages

Sangraha — 251B Token Indic Language Pretraining Corpus (22 Languages)

Category
Text
Records
177,355,164 rows
Format
PARQUET
Update Frequency
One-time snapshot
Collection Method
auto_imported_huggingface_federated
PII
None detected
File Size
~441628.01 MB
Downloads
0

About this data

Largest cleaned Indic-language pretraining dataset: 251B tokens across 22 Indian languages, curated from web sources, multilingual corpora, and large-scale translations. CC-BY-4.0, parquet format.

Schema

NameTypeDescription
doc_idVARCHARUnique SHA-1 hash identifier for each document in the corpus.
typeVARCHARContent source type: 'web', 'synthetic', or other origin category.
textVARCHARRaw text document content in Indic languages or English, may contain multiple languages within single document.

Sample Data

Preview a sample of the data before downloading.

Free

Open dataset

Quality: No ratings
0 downloads
Seller: DataBazaar
Sign up to download

Agent? No sign-up needed →

For AI Agents

Via MCP Server
# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
  "mcpServers": {
    "databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
  }
}

# 2. Your agent can then call:
search_datasets({ query: "Sangraha — 251B Token Indic La" })
// Found: 1275f2a9-9b18-475d-8466-e45784041829
get_download_url({ dataset_id: "1275f2a9-9b18-475d-8466-e45784041829" })  // free — no API key needed
Via REST API
# Free dataset — no API key required:
curl https://api.databazaar.io/datasets/1275f2a9-9b18-475d-8466-e45784041829/download-url