DynaBERT: Dynamic BERT with Adaptive Width and Depth
DynaBERT can flexibly adjust the size and latency by selecting adaptive width and depth, and
the subnetworks of it have competitive performances as other similar-sized compressed models.
The training process of DynaBERT includes first training a width-adaptive BERT and then
allowing both adaptive width and depth using knowledge distillation.
This code is modified based on the repository developed by Hugging Face:
Transformers v2.1.1
, and is released in
GitHub
.
@inproceedings{hou2020dynabert,
title = {DynaBERT: Dynamic BERT with Adaptive Width and Depth},
author = {Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, Qun Liu},
booktitle = {Advances in Neural Information Processing Systems},
year = {2020}
}
Runs of huawei-noah DynaBERT_SST-2 on huggingface.co
12
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
-4
30-day runs
More Information About DynaBERT_SST-2 huggingface.co Model
DynaBERT_SST-2 huggingface.co
DynaBERT_SST-2 huggingface.co is an AI model on huggingface.co that provides DynaBERT_SST-2's model effect (), which can be used instantly with this huawei-noah DynaBERT_SST-2 model. huggingface.co supports a free trial of the DynaBERT_SST-2 model, and also provides paid use of the DynaBERT_SST-2. Support call DynaBERT_SST-2 model through api, including Node.js, Python, http.
DynaBERT_SST-2 huggingface.co is an online trial and call api platform, which integrates DynaBERT_SST-2's modeling effects, including api services, and provides a free online trial of DynaBERT_SST-2, you can try DynaBERT_SST-2 online for free by clicking the link below.
huawei-noah DynaBERT_SST-2 online free url in huggingface.co:
DynaBERT_SST-2 is an open source model from GitHub that offers a free installation service, and any user can find DynaBERT_SST-2 on GitHub to install. At the same time, huggingface.co provides the effect of DynaBERT_SST-2 install, users can directly use DynaBERT_SST-2 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.