Deep RL from Human Preferences (Foundational RLHF)
A 2017 paper showing that you could train a reinforcement learning agent using only humans ranking which of two behaviors looked better — no need to hand-design a reward function. The technique sat largely unused in language modeling until 2022, when InstructGPT applied it at scale and it became the dominant alignment recipe for every modern chatbot.
Why it matters
The bridge between "you can train AI via reward signals" (classical RL, 1990s-2010s) and "you can train AI via human judgment" (LLM alignment, 2022+). Without this paper, ChatGPT's launch is architecturally impossible.
Core Capabilities
Research
Foundational paper or scientific contribution.
Multimodal
Combines text, vision, and audio in one model.
Context Window
Context window not disclosed.
Availability
API
Not available
Product / App
Not available
Open Source
Not released
Enterprise
—
Pricing Model
Research artifact
Not commercially released.
Research Capability / Performance
Where this model sits relative to the middle 60% of models in the tree. All scores are 0–10 (higher is better).
Lower 20% Upper 80% This model
Lower 20% 20th percentile — 20% of models score below this This model Where the current model lands Upper 80% 80th percentile — only 20% of models score above this
Percentile boundaries are computed across every model in the tree that reports the underlying benchmark for each capability.
What it feels like
- Language model from OpenAI — see the linked sources below for benchmark and review coverage
Best use cases
- General-purpose tasks within OpenAI's deployment footprint
- See the model spec and sources block for benchmarked use cases
Tools to try
Not ideal for
- Tasks far outside the modalities listed in this model's spec
- Workflows where a more recent successor in the same family scores higher