textSciCodePile/SciCode-Domain-Codecodescientific-computinggithubbiologychemistryphysicsmaterials-sciencellm-pretrainingdomain-specificapache-2.0

SciCode Scientific Domain Code

Name: SciCode Scientific Domain Code
License: https://huggingface.co/datasets/SciCodePile/SciCode-Domain-Code

Free

Open dataset

Sample structure: 100 / 100

1 download links issued

Seller: DataBazaar

Already have an account? Log in

Agent? Connect your account →

Source, license and coverage

Supplier documentation. These claims are separate from the automated sample score. A listing edit date is not a data freshness date.

License: apache-2.0
Source / creator: SciCodePile/SciCode-Domain-Code
Collection method: The maintainers crawled public GitHub repositories tagged or classified into 178 interdisciplinary scientific topics, extracted source files, and organized them into one CSV per domain. Files preserve original source contents along with repo and path metadata. No additional deduplication or quality filtering is documented beyond domain-topic grouping.
Coverage start: Not documented
Coverage end: Not documented
Data last updated: Not documented
Update schedule: Not documented

Source documentation ↗

License terms ↗

The source does not document deduplication, license-per-file verification, PII scrubbing, or quality filtering procedures in detail. Upstream GitHub repos carry heterogeneous individual licenses that may differ from the dataset's Apache-2.0 aggregate license — downstream users training models should verify per-file licenses for compliance. Domain labels are topic-based and may be noisy. Coverage skews toward repos with English documentation and topics popular on GitHub. Buyers should validate empirically for their use case.

Sample structure score: 100 / 100

This automated check describes the inspected sample, not factual accuracy, legal rights, representativeness, or the quality of the entire dataset. It is not a customer rating.

Assessed 10 sample records (JSON) on 2026-10-09. All records in the provided sample were checked.

Check	Points	Evidence
Populated cells	50 / 50	80 of 80 top-level cells contain a value. Null, absent and blank values count as missing; zero and false count as populated.
Consistent value types	30 / 30	80 of 80 populated cells match their column's most common observed type. Types are inferred, not checked against real-world truth.
Consistent record shape	20 / 20	10 of 10 records have the expected fields. CSV/TSV use the header width; JSON uses the union of observed keys.

Field-level findings and improvements

Check missing cells and mixed types below. Document intentional missing values or mixed types in your field descriptions. Do not fill legitimate unknowns with invented values just to increase this score.

Field	Missing cells	Most common type	Other populated types
keyword	0 / 10	string	0 / 10
repo_name	0 / 10	string	0 / 10
file_path	0 / 10	string	0 / 10
file_extension	0 / 10	string	0 / 10
file_size	0 / 10	number	0 / 10
line_count	0 / 10	number	0 / 10
content	0 / 10	string	0 / 10
language	0 / 10	string	0 / 10

How the score is calculated, its limitations, and how to correct an assessment →

About this data

Scientific source code from GitHub spanning biology, chemistry, materials science, physics, and 174 additional domains. Covers 155,855 code repositories with 1.1 billion+ lines of code across approximately 115 GB.

Retrieve with your agent or Python

Create an account and configure DATABAZAAR_API_KEY. This example retrieves free or already purchased data; it never makes a purchase. For a multi-file dataset, choose a file index from the manifest.

Download the Python example

python3 retrieve-dataset.py be5165cb-678d-4abf-97d4-74626aa538d7 --output dataset.bin

Full supplier documentation

## Overview SciCode-Domain-Code (DATA1) is a large-scale domain-specific code corpus aggregated from GitHub repositories, focused on interdisciplinary scientific computing. It spans 178 domain topics (biology, chemistry, materials science, physics, etc.) totaling ~115 GB and over 1.1 billion lines of code across 1M-10M rows. Distributed as 178 CSV files, one per domain topic. ## Schema The dataset is split into 178 per-domain CSV files. Typical columns (per HF card and standard code-corpus conventions): - repo_name — string — source GitHub repository identifier - file_path — string — path of the source file within the repo - content — string — raw source code contents - language — string — programming language - license — string — repo license metadata where available - size — int — file size in bytes - domain — string — scientific domain label (one of 178) - stars / forks — int — repo popularity signals (where present) Exact column set may vary slightly per CSV; consult individual files. ## Sources - SciCodePile/SciCode-Domain-Code on Hugging Face — https://huggingface.co/datasets/SciCodePile/SciCode-Domain-Code — Apache-2.0 - Upstream: public GitHub repositories filtered by scientific-domain topics ## Methodology The maintainers crawled public GitHub repositories tagged or classified into 178 interdisciplinary scientific topics, extracted source files, and organized them into one CSV per domain. Files preserve original source contents along with repo and path metadata. No additional deduplication or quality filtering is documented beyond domain-topic grouping. ## Known gaps & limitations The source does not document deduplication, license-per-file verification, PII scrubbing, or quality filtering procedures in detail. Upstream GitHub repos carry heterogeneous individual licenses that may differ from the dataset's Apache-2.0 aggregate license — downstream users training models should verify per-file licenses for compliance. Domain labels are topic-based and may be noisy. Coverage skews toward repos with English documentation and topics popular on GitHub. Buyers should validate empirically for their use case. ## Intended use & out-of-scope - IS for: pretraining/fine-tuning code LLMs on scientific computing, RAG over domain-specific code, building domain code-search and code-completion tools, studying scientific software patterns. - NOT for: production deployment without license review of underlying repos; not deduplicated against common code eval suites (HumanEval, MBPP, etc.) — leakage risk for benchmark training. _Federated dataset: 7 parquet shards, 3.20 GB total. Queries and downloads stream through the DataBazaar API._ Original supplier listing: SciCode Domain Code: 1.1B+ Lines of Scientific Code Across 178 Domains Large-scale domain-specific code dataset (~115 GB, 1.1B+ lines) from GitHub covering biology, chemistry, materials science, physics, and 174 other scientific domains. Apache 2.0 licensed.

Schema

Name	Type	Description
keyword	VARCHAR	Scientific domain or topic label (e.g., '3D', 'biology', 'chemistry')
repo_name	VARCHAR	GitHub repository identifier in owner/name format
file_path	VARCHAR	Full path to source file within repository
file_extension	VARCHAR	File extension including dot (e.g., '.py', '.cpp', '.java')
file_size	BIGINT	File size in bytes
line_count	BIGINT	Number of lines of code in file
content	VARCHAR	Raw source code contents as text
language	VARCHAR	Programming language name (e.g., 'Python', 'C++', 'Java')

Sample Data

Preview a sample of the data before downloading.

Public sample only. Sign in to retrieve the full dataset, including free datasets.

For AI Agents

Via MCP Server

# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
  "mcpServers": {
    "databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
  }
}

# 2. Your agent can then call:
search_datasets({ query: "SciCode Scientific Domain Code" })
// Found: be5165cb-678d-4abf-97d4-74626aa538d7
get_download_url({ dataset_id: "be5165cb-678d-4abf-97d4-74626aa538d7" })  // free — sign in with MCP OAuth first

Via REST API

# Free dataset — sign in or use your account API key:
curl https://api.databazaar.io/datasets/be5165cb-678d-4abf-97d4-74626aa538d7/download-url -H "Authorization: Bearer $DATABAZAAR_API_KEY"