Llama 3.1 Series
Llama 3.1 is an API model from Nvidia. It’s positioned for hard reasoning and planning—work that benefits from iteration, not just one-shot answers.
Intelligence
Below avg
Speed
Good
158 tok/s output
Cost
Low
$0.10 in / $0.10 out
Context
128K
Up to 128,000 tokens
How are Intelligence, Speed & Cost bucketed?
Intelligence and Speed buckets are percentile ranks on
Artificial Analysis. Cost buckets are fixed dollar
thresholds keyed off output-token price ($/M out).
Intelligence
- Top 1%≤ 1%
- Top 5%≤ 5%
- Top 10%≤ 10%
- Good≤ 25%
- Medium≤ 50%
- Below avg> 50%
Speed
- Top 1%≥ 345 tok/s
- Top 5%≥ 237 tok/s
- Top 10%≥ 196 tok/s
- Good≥ 146 tok/s
- Medium≥ 90 tok/s
- Slow< 90 tok/s
Cost
- Freeopen weights · self-host
- Low< $1 / M out
- Moderate$1–5 / M out
- High≥ $5 / M out
Core Capabilities
Long Documents
Handles entire codebases, books, and multi-doc RAG.
Reasoning
Solves complex math, logic, and planning tasks.
Context Window
128k tokens
≈ 98 pages
4k Chat 聊天
32k Long docs 长文档
128k This model 本模型
400k Multi-doc 多文档
1M Codebase 整个代码库
10M
Availability
API
Available
Product / App
Not available
Open Source
Not released
Enterprise
Contact sales
Pricing Model
Free / self-host
Open weights — pay only for compute.
Self-host Capability / Performance
Where this model sits relative to the middle 60% of models in the tree. All scores are 0–10 (higher is better).
Lower 20% Upper 80% This model
Context / memory
Context window size · log-scaled
6.0
9.0
6.0
Lower 20% 20th percentile — 20% of models score below this This model Where the current model lands Upper 80% 80th percentile — only 20% of models score above this
Percentile boundaries are computed across every model in the tree that reports the underlying benchmark for each capability.
What it feels like
- Llama 3.1 405B was the world's largest openly available foundation model at release
- Trained on 15T tokens across 16K+ H100 GPUs — Meta opted for dense decoder over MoE for training stability
- MMLU 87.3% (Instruct) — matched or beat GPT-4-Turbo (86.5%) and Claude 3 Opus (86.8%)
- MATH 73.8% — beaten only by GPT-4o (76.6%); edged GPT-4T and Claude 3.5 Sonnet
- 128K context, native tool use, multilingual coverage in 8 languages
- License change allowed using Llama outputs to train other models — kicked off the synthetic-data wave
Best use cases
- Self-hosted frontier-quality LLM deployments where you can spare ~8 H100s
- Distillation source for smaller open models (license-permissive synthetic data)
- Multilingual production work in the 8 supported languages
- Tool-using agent workflows where open weights matter
Tools to try
Not ideal for
- Single-GPU / edge deployments — 405B is too big without aggressive quantization
- Frontier reasoning by 2025 — Llama 3.3 70B and Llama 4 family supersede it
- Multimodal tasks (Llama 3.1 base is text-only; vision came in 3.2)