Method evidence record
Knowledge Distillation
Knowledge Distillation is a model-compression technique, introduced by Geoffrey Hinton and colleagues in 2015, that trains a small student model using the soft-label outputs of a large teacher model. Distilled models such as DistilBERT and TinyBERT reach roughly 97% of the larger model's performance while running far faster.
Source record
Citations copied verbatim from the method’s source record. No claim-level verification is inferred from them.
Knowledge Distillation (Teacher–Student Model Compression)
Taxonomic method record · ml-model / deep-learning
- Hinton, G., Vinyals, O. & Dean, J. (2015). Distilling the Knowledge in a Neural Network. NeurIPS Deep Learning Workshop. · URL
- Sanh, V., Debut, L., Chaumond, J. & Wolf, T. (2019). DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv:1910.01108. · URL
Curated claims
Claims persisted in the evidence ledger, each with its own assessment.
No curated claims yet
This view does not invent a claim assessment when the ledger has none.
Related methods
Generated from the method graph and shown as machine-suggested relations — no evidence claim is inferred.