textBramVanroy/CommonCrawl-CreativeCommons-finecommon-crawlcreative-commonsmultilingualllm-pretrainingfinewebweb-textlanguage-modelingparquet

Common Crawl Creative Commons Fine (C5f) — Multilingual High-Quality Web Corpus

Name: Common Crawl Creative Commons Fine (C5f) — Multilingual High-Quality Web Corpus
Creator: DataBazaar
Keywords: BramVanroy/CommonCrawl-CreativeCommons-fine, common-crawl, creative-commons, multilingual, llm-pretraining, fineweb, web-text, language-modeling, parquet

About this data

Filtered Creative Commons web corpus from Common Crawl, intersected with FineWeb/FineWeb-2 for quality. Multilingual (EN, DE, FR, NL, ES, IT, AF, FY), 10M-100M rows, Parquet format. Ideal for LLM pretraining and fine-tuning.

Schema

Name	Type	Description
text	VARCHAR
id	VARCHAR
dump	VARCHAR
url	VARCHAR
date	VARCHAR
file_path	VARCHAR
license_abbr	VARCHAR
license_version	VARCHAR
license_location	VARCHAR
license_in_head	BOOLEAN
license_in_footer	BOOLEAN
license_element	VARCHAR
license_left_context	VARCHAR
license_right_context	VARCHAR
potential_licenses	STRUCT(abbr VARCHAR[], in_footer BOOLEAN[], in_head BOOLEAN[], "location" VARCHAR[], "version" VARCHAR[], element VARCHAR[], left_context VARCHAR[], right_context VARCHAR[])
license_parse_error	BOOLEAN
license_disagreement	BOOLEAN
language_script	VARCHAR
language	VARCHAR
language_score	DOUBLE

Sample Data

Preview a sample of the data before downloading.

Free

Open dataset

Quality: No ratings

0 downloads

Seller: DataBazaar

Agent? No sign-up needed →

For AI Agents

Via MCP Server

# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
  "mcpServers": {
    "databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
  }
}

# 2. Your agent can then call:
search_datasets({ query: "Common Crawl Creative Commons " })
// Found: 61ac2c8d-9c3a-4a35-8d10-1158831cc41d
get_download_url({ dataset_id: "61ac2c8d-9c3a-4a35-8d10-1158831cc41d" })  // free — no API key needed

Via REST API

# Free dataset — no API key required:
curl https://api.databazaar.io/datasets/61ac2c8d-9c3a-4a35-8d10-1158831cc41d/download-url