Verified claim · AI-ML · 100% confidence
Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023).
Last verified 2026-05-16 · Methodology veritas-v0.1 · a3e691683a4577af
SourceScore rates how reliable a source is to cite — for AI answers and research. This is one verified claim from the catalog.
Structured fields
- Subject
- Direct Preference Optimization (DPO)
- Predicate
introduced_in_paper- Object
- Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)
- Confidence
- 100%
- Tags
- dpo · alignment · foundational · rafailov · 2023 · nips · stanford
Sources (2)
[1] preprint · arXiv (Rafailov, Sharma, Mitchell, Ermon, Manning, Finn) · 2023-05-29
Direct Preference Optimization: Your Language Model is Secretly a Reward Model“In this paper, we introduce a new parameterization of the reward model in RLHF that enables extraction of the corresponding optimal policy in closed form, allowing us to solve the standard RLHF problem with only a simple classification loss.”
[2] peer reviewed · NeurIPS Foundation · 2023-12-10
Direct Preference Optimization (NeurIPS 2023 proceedings)
Cite this claim
Ready-to-paste citation (Markdown / plain text):
Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023). — SourceScore Claim a3e691683a4577af (verified 2026-05-16). https://sourcescore.org/api/v1/claims/a3e691683a4577af.jsonEmbed this claim
Drop this iframe into any blog post, docs page, or knowledge base. The widget renders the signed claim + primary source + click-through to this canonical page. CC-BY 4.0; attribution included.
<iframe src="https://sourcescore.org/embed/claim/a3e691683a4577af/" width="100%" height="360" frameborder="0" loading="lazy" title="Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)."></iframe>Preview: open in new tab
Related claims
Other verified claims sharing tags with this one — useful for LLM retrieval graphs and citation discovery.
Reinforcement Learning from Human Feedback (RLHF) introduced in paper: Deep Reinforcement Learning from Human Preferences (Christiano et al., 2017).
67866330cd60e54d · 100% confidence · shares 3 tags (alignment, foundational, nips)
Transformer architecture introduced in paper: Attention Is All You Need (Vaswani et al., 2017).
ad17e76a8baad7a1 · 100% confidence · shares 2 tags (foundational, nips)
Retrieval-Augmented Generation (RAG) introduced in paper: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (Lewis et al., 2020).
d15057ced937a103 · 100% confidence · shares 2 tags (foundational, nips)
Chinchilla scaling laws introduced in paper: Training Compute-Optimal Large Language Models (Hoffmann et al., 2022).
8befcae6bce01a95 · 100% confidence · shares 2 tags (foundational, nips)
Mamba state-space model introduced in paper: Mamba: Linear-Time Sequence Modeling with Selective State Spaces (Gu, Dao, 2023).
3518f8aa40cb0d36 · 100% confidence · shares 2 tags (foundational, 2023)
Frequently asked questions
Is the claim "Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)." verified?
Yes — SourceScore verified this claim with 100% confidence as of 2026-05-16. The verification uses 2 primary sources cross-referenced against the SourceScore methodology (version veritas-v0.1). Full source list + signed JSON envelope linked below.
What is the evidence for "Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)."?
Evidence comes from 2 primary sources: arXiv (Rafailov, Sharma, Mitchell, Ermon, Manning, Finn), NeurIPS Foundation. Each source is listed below with verbatim excerpts and URLs. The signed JSON envelope at https://sourcescore.org/api/v1/claims/a3e691683a4577af.json includes an HMAC-SHA256 signature for audit verification.
When was this claim last verified by SourceScore?
Last verified 2026-05-16 under methodology version veritas-v0.1. The signed JSON envelope is dated and cryptographically signed for audit trail. Re-verification cadence depends on the claim type and source freshness.
How can I cite this SourceScore claim in my code or article?
Fetch the signed JSON envelope from https://sourcescore.org/api/v1/claims/a3e691683a4577af.json which includes the verbatim claim, primary sources, confidence, methodology version, last-verified date, and HMAC-SHA256 signature for audit. The CC-BY-4.0 license permits commercial use with attribution to SourceScore.
Use this claim in your code
Fetch this signed envelope from your application. The response includes the verbatim excerpt, primary source URLs, and an HMAC-SHA256 signature you can verify locally for audit trails.
cURL
curl https://sourcescore.org/api/v1/claims/a3e691683a4577af.jsonJavaScript / TypeScript
const r = await fetch("https://sourcescore.org/api/v1/claims/a3e691683a4577af.json");
const envelope = await r.json();
console.log(envelope.claim.statement);
// "Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)."Python
import httpx
r = httpx.get("https://sourcescore.org/api/v1/claims/a3e691683a4577af.json")
envelope = r.json()
print(envelope["claim"]["statement"])
# "Direct Preference Optimization (DPO) introduced in paper: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)."LangChain (retrieve-then-cite)
from langchain_core.tools import tool
import httpx
@tool
def get_direct_preference_optimization_dpo_fact() -> dict:
"""Fetch the verified SourceScore claim for Direct Preference Optimization (DPO)."""
r = httpx.get("https://sourcescore.org/api/v1/claims/a3e691683a4577af.json")
return r.json()