textopenbmb/UltraFeedbackrlhfpreference-datareward-modeldpoalignmentllmfine-tuninginstruction-followinggpt-4-judgeopenbmb

UltraFeedback — Large-Scale Fine-Grained Preference Dataset

Name: UltraFeedback — Large-Scale Fine-Grained Preference Dataset
Creator: DataBazaar
Keywords: openbmb/UltraFeedback, rlhf, preference-data, reward-model, dpo, alignment, llm, fine-tuning, instruction-following, gpt-4-judge, openbmb

About this data

64k prompts × 4 LLM responses (256k samples) with fine-grained GPT-4 preference annotations across instruction-following, truthfulness, honesty, and helpfulness. Canonical dataset for reward model and RLHF training.

Schema

Name	Type	Description
source	VARCHAR	Origin of the prompt (UltraChat, ShareGPT, Evol-Instruct, TruthfulQA, FalseQA, FLAN)
instruction	VARCHAR	User prompt or task description
models	VARCHAR[]	Names of 4 LLMs that generated completions for this instruction
completions	STRUCT(annotations STRUCT(helpfulness STRUCT(Rating VARCHAR, Rationale VARCHAR, "Rationale For Rating" VARCHAR, "Type" VARCHAR[]), honesty STRUCT(Rating VARCHAR, Rationale VARCHAR), instruction_following STRUCT(Rating VARCHAR, Rationale VARCHAR), truthfulness STRUCT(Rating VARCHAR, Rationale VARCHAR, "Rationale For Rating" VARCHAR, "Type" VARCHAR[])), critique VARCHAR, custom_system_prompt VARCHAR, "fine-grained_score" DOUBLE, model VARCHAR, overall_score DOUBLE, principle VARCHAR, response VARCHAR)[]	List of response records with model name, generated text, GPT-4 annotations (instruction-following, truthfulness, honesty, helpfulness ratings + rationales), overall score, and critique
correct_answers	VARCHAR[]	Array of reference correct answers for evaluation
incorrect_answers	VARCHAR[]	Array of reference incorrect answers for evaluation

Sample Data

Preview a sample of the data before downloading.

Free

Open dataset

Quality: No ratings

0 downloads

Seller: DataBazaar

Agent? No sign-up needed →

For AI Agents

Via MCP Server

# 1. Add to your agent's MCP config (claude_desktop_config.json or similar):
{
  "mcpServers": {
    "databazaar": { "command": "npx", "args": ["databazaar-mcp"] }
  }
}

# 2. Your agent can then call:
search_datasets({ query: "UltraFeedback — Large-Scale Fi" })
// Found: 9d3edf8f-9074-4178-bb35-e25c2e645b5f
get_download_url({ dataset_id: "9d3edf8f-9074-4178-bb35-e25c2e645b5f" })  // free — no API key needed

Via REST API

# Free dataset — no API key required:
curl https://api.databazaar.io/datasets/9d3edf8f-9074-4178-bb35-e25c2e645b5f/download-url