DistilBERT introduced in: Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation.
Object
Sanh et al. 2019 — a smaller, faster, cheaper BERT via knowledge distillation
Primary source · preprint · 2019-10-02
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter — arXiv (Sanh, Debut, Chaumond, Wolf / Hugging Face)