Verified claim · AI-ML · 100% confidence
DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation.
Last verified 2026-05-16 · Methodology veritas-v0.1 · 245af747a3d21061
SourceScore rates how reliable a source is to cite — for AI answers and research. This is one verified claim from the catalog.
Structured fields
- Subject
- DistilBERT
- Predicate
introduced_in- Object
- Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation
- Confidence
- 100%
- Tags
- distilbert · bert · knowledge-distillation · hugging-face · foundational · 2019 · introduced_in
Sources (2)
[1] preprint · arXiv (Sanh, Debut, Chaumond, Wolf / Hugging Face) · 2019-10-02
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter“We introduce a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. We show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster.”
[2] official blog · Hugging Face · 2019-10-02
DistilBERT — Hugging Face Transformers documentationHugging Face is rated by SourceScore — see its reliability →
Cite this claim
Ready-to-paste citation (Markdown / plain text):
DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation. — SourceScore Claim 245af747a3d21061 (verified 2026-05-16). https://sourcescore.org/api/v1/claims/245af747a3d21061.jsonEmbed this claim
Drop this iframe into any blog post, docs page, or knowledge base. The widget renders the signed claim + primary source + click-through to this canonical page. CC-BY 4.0; attribution included.
<iframe src="https://sourcescore.org/embed/claim/245af747a3d21061/" width="100%" height="360" frameborder="0" loading="lazy" title="DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation."></iframe>Preview: open in new tab
Related claims
Other verified claims sharing tags with this one — useful for LLM retrieval graphs and citation discovery.
RoBERTa introduced in: Liu et al. 2019 — A Robustly Optimized BERT Pretraining Approach.
d4fecb26a4c9cdca · 100% confidence · shares 4 tags (bert, foundational, 2019…)
BART introduced in: Lewis et al. 2019 — denoising sequence-to-sequence pretraining.
f5b422e3255fd7c0 · 100% confidence · shares 3 tags (foundational, 2019, introduced_in)
Knowledge Distillation popularized in: Hinton, Vinyals, Dean 2015 — distilling the knowledge in a neural network.
f14acb906ba6c12f · 100% confidence · shares 3 tags (knowledge-distillation, foundational, introduced_in)
ARC-AGI benchmark introduced in: Chollet 2019 — abstraction and reasoning corpus.
cc5df3c14d35fa49 · 100% confidence · shares 3 tags (foundational, 2019, introduced_in)
PEFT (parameter-efficient fine-tuning) popularized in: Houlsby et al. 2019 — Adapter Modules + downstream PEFT library.
fb9a06ffca4277c1 · 100% confidence · shares 3 tags (foundational, 2019, introduced_in)
Frequently asked questions
Is the claim "DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation." verified?
Yes — SourceScore verified this claim with 100% confidence as of 2026-05-16. The verification uses 2 primary sources cross-referenced against the SourceScore methodology (version veritas-v0.1). Full source list + signed JSON envelope linked below.
What is the evidence for "DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation."?
Evidence comes from 2 primary sources: arXiv (Sanh, Debut, Chaumond, Wolf / Hugging Face), Hugging Face. Each source is listed below with verbatim excerpts and URLs. The signed JSON envelope at https://sourcescore.org/api/v1/claims/245af747a3d21061.json includes an HMAC-SHA256 signature for audit verification.
When was this claim last verified by SourceScore?
Last verified 2026-05-16 under methodology version veritas-v0.1. The signed JSON envelope is dated and cryptographically signed for audit trail. Re-verification cadence depends on the claim type and source freshness.
How can I cite this SourceScore claim in my code or article?
Fetch the signed JSON envelope from https://sourcescore.org/api/v1/claims/245af747a3d21061.json which includes the verbatim claim, primary sources, confidence, methodology version, last-verified date, and HMAC-SHA256 signature for audit. The CC-BY-4.0 license permits commercial use with attribution to SourceScore.
Use this claim in your code
Fetch this signed envelope from your application. The response includes the verbatim excerpt, primary source URLs, and an HMAC-SHA256 signature you can verify locally for audit trails.
cURL
curl https://sourcescore.org/api/v1/claims/245af747a3d21061.jsonJavaScript / TypeScript
const r = await fetch("https://sourcescore.org/api/v1/claims/245af747a3d21061.json");
const envelope = await r.json();
console.log(envelope.claim.statement);
// "DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation."Python
import httpx
r = httpx.get("https://sourcescore.org/api/v1/claims/245af747a3d21061.json")
envelope = r.json()
print(envelope["claim"]["statement"])
# "DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation."LangChain (retrieve-then-cite)
from langchain_core.tools import tool
import httpx
@tool
def get_distilbert_fact() -> dict:
"""Fetch the verified SourceScore claim for DistilBERT."""
r = httpx.get("https://sourcescore.org/api/v1/claims/245af747a3d21061.json")
return r.json()