microsoft / MiniLM-L12-H384-uncased

huggingface.co
Total runs: 35.0K
24-hour runs: 0
7-day runs: -1.3K
30-day runs: 5.2K
Model's Last Updated: May 20 2021
text-classification

Introduction of MiniLM-L12-H384-uncased

Model Details of MiniLM-L12-H384-uncased

MiniLM: Small and Fast Pre-trained Models for Language Understanding and Generation

MiniLM is a distilled model from the paper " MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers ".

Please find the information about preprocessing, training and full details of the MiniLM in the original MiniLM repository .

Please note: This checkpoint can be an inplace substitution for BERT and it needs to be fine-tuned before use!

English Pre-trained Models

We release the uncased 12 -layer model with 384 hidden size distilled from an in-house pre-trained UniLM v2 model in BERT-Base size.

  • MiniLMv1-L12-H384-uncased: 12-layer, 384-hidden, 12-heads, 33M parameters, 2.7x faster than BERT-Base
Fine-tuning on NLU tasks

We present the dev results on SQuAD 2.0 and several GLUE benchmark tasks.

Model #Param SQuAD 2.0 MNLI-m SST-2 QNLI CoLA RTE MRPC QQP
BERT-Base 109M 76.8 84.5 93.2 91.7 58.9 68.6 87.3 91.3
MiniLM-L12xH384 33M 81.7 85.7 93.0 91.5 58.5 73.3 89.5 91.3
Citation

If you find MiniLM useful in your research, please cite the following paper:

@misc{wang2020minilm,
    title={MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers},
    author={Wenhui Wang and Furu Wei and Li Dong and Hangbo Bao and Nan Yang and Ming Zhou},
    year={2020},
    eprint={2002.10957},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

Runs of microsoft MiniLM-L12-H384-uncased on huggingface.co

35.0K
Total runs
0
24-hour runs
469
3-day runs
-1.3K
7-day runs
5.2K
30-day runs

More Information About MiniLM-L12-H384-uncased huggingface.co Model

More MiniLM-L12-H384-uncased license Visit here:

https://choosealicense.com/licenses/mit

MiniLM-L12-H384-uncased huggingface.co

MiniLM-L12-H384-uncased huggingface.co is an AI model on huggingface.co that provides MiniLM-L12-H384-uncased's model effect (), which can be used instantly with this microsoft MiniLM-L12-H384-uncased model. huggingface.co supports a free trial of the MiniLM-L12-H384-uncased model, and also provides paid use of the MiniLM-L12-H384-uncased. Support call MiniLM-L12-H384-uncased model through api, including Node.js, Python, http.

MiniLM-L12-H384-uncased huggingface.co Url

https://huggingface.co/microsoft/MiniLM-L12-H384-uncased

microsoft MiniLM-L12-H384-uncased online free

MiniLM-L12-H384-uncased huggingface.co is an online trial and call api platform, which integrates MiniLM-L12-H384-uncased's modeling effects, including api services, and provides a free online trial of MiniLM-L12-H384-uncased, you can try MiniLM-L12-H384-uncased online for free by clicking the link below.

microsoft MiniLM-L12-H384-uncased online free url in huggingface.co:

https://huggingface.co/microsoft/MiniLM-L12-H384-uncased

MiniLM-L12-H384-uncased install

MiniLM-L12-H384-uncased is an open source model from GitHub that offers a free installation service, and any user can find MiniLM-L12-H384-uncased on GitHub to install. At the same time, huggingface.co provides the effect of MiniLM-L12-H384-uncased install, users can directly use MiniLM-L12-H384-uncased installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

MiniLM-L12-H384-uncased install url in huggingface.co:

https://huggingface.co/microsoft/MiniLM-L12-H384-uncased

Url of MiniLM-L12-H384-uncased

MiniLM-L12-H384-uncased huggingface.co Url

Provider of MiniLM-L12-H384-uncased huggingface.co

microsoft
ORGANIZATIONS

Other API from microsoft

huggingface.co

Total runs: 681.1K
Run Growth: 208.0K
Growth Rate: 30.53%
Updated:February 03 2022
huggingface.co

Total runs: 595.8K
Run Growth: -131.1K
Growth Rate: -22.00%
Updated:November 25 2025
huggingface.co

Total runs: 535.6K
Run Growth: 307.0K
Growth Rate: 57.32%
Updated:April 08 2024
huggingface.co

Total runs: 511.8K
Run Growth: -531.5K
Growth Rate: -103.84%
Updated:December 08 2025
huggingface.co

Total runs: 289.1K
Run Growth: -22.3K
Growth Rate: -7.70%
Updated:February 14 2024
huggingface.co

Total runs: 117.8K
Run Growth: -147.4K
Growth Rate: -125.12%
Updated:November 08 2023
huggingface.co

Total runs: 117.6K
Run Growth: -1.2K
Growth Rate: -1.03%
Updated:February 29 2024
huggingface.co

Total runs: 100.0K
Run Growth: -425
Growth Rate: -0.42%
Updated:August 28 2025
huggingface.co

Total runs: 69.7K
Run Growth: -13.1K
Growth Rate: -18.74%
Updated:November 25 2025
huggingface.co

Total runs: 68.7K
Run Growth: -18.2K
Growth Rate: -26.47%
Updated:December 03 2025
huggingface.co

Total runs: 47.4K
Run Growth: -4.9K
Growth Rate: -10.34%
Updated:December 23 2021
huggingface.co

Total runs: 40.6K
Run Growth: -238.3K
Growth Rate: -586.57%
Updated:September 26 2022
huggingface.co

Total runs: 33.0K
Run Growth: -16.2K
Growth Rate: -48.91%
Updated:May 12 2026
huggingface.co

Total runs: 32.5K
Run Growth: 28.2K
Growth Rate: 86.85%
Updated:April 23 2026
huggingface.co

Total runs: 23.4K
Run Growth: -106.1K
Growth Rate: -454.00%
Updated:February 03 2023
huggingface.co

Total runs: 20.4K
Run Growth: -9.1K
Growth Rate: -44.40%
Updated:April 24 2023