valhalla / distilbart-mnli-12-3

huggingface.co
Total runs: 50.4K
24-hour runs: -135
7-day runs: -843
30-day runs: 40.8K
Model's Last Updated: June 14 2021
zero-shot-classification

Introduction of distilbart-mnli-12-3

Model Details of distilbart-mnli-12-3

DistilBart-MNLI

distilbart-mnli is the distilled version of bart-large-mnli created using the No Teacher Distillation technique proposed for BART summarisation by Huggingface, here .

We just copy alternating layers from bart-large-mnli and finetune more on the same data.

matched acc mismatched acc
bart-large-mnli (baseline, 12-12) 89.9 90.01
distilbart-mnli-12-1 87.08 87.5
distilbart-mnli-12-3 88.1 88.19
distilbart-mnli-12-6 89.19 89.01
distilbart-mnli-12-9 89.56 89.52

This is a very simple and effective technique, as we can see the performance drop is very little.

Detailed performace trade-offs will be posted in this sheet .

Fine-tuning

If you want to train these models yourself, clone the distillbart-mnli repo and follow the steps below

Clone and install transformers from source

git clone https://github.com/huggingface/transformers.git
pip install -qqq -U ./transformers

Download MNLI data

python transformers/utils/download_glue_data.py --data_dir glue_data --tasks MNLI

Create student model

python create_student.py \
  --teacher_model_name_or_path facebook/bart-large-mnli \
  --student_encoder_layers 12 \
  --student_decoder_layers 6 \
  --save_path student-bart-mnli-12-6 \

Start fine-tuning

python run_glue.py args.json

You can find the logs of these trained models in this wandb project .

Runs of valhalla distilbart-mnli-12-3 on huggingface.co

50.4K
Total runs
-135
24-hour runs
-284
3-day runs
-843
7-day runs
40.8K
30-day runs

More Information About distilbart-mnli-12-3 huggingface.co Model

distilbart-mnli-12-3 huggingface.co

distilbart-mnli-12-3 huggingface.co is an AI model on huggingface.co that provides distilbart-mnli-12-3's model effect (), which can be used instantly with this valhalla distilbart-mnli-12-3 model. huggingface.co supports a free trial of the distilbart-mnli-12-3 model, and also provides paid use of the distilbart-mnli-12-3. Support call distilbart-mnli-12-3 model through api, including Node.js, Python, http.

distilbart-mnli-12-3 huggingface.co Url

https://huggingface.co/valhalla/distilbart-mnli-12-3

valhalla distilbart-mnli-12-3 online free

distilbart-mnli-12-3 huggingface.co is an online trial and call api platform, which integrates distilbart-mnli-12-3's modeling effects, including api services, and provides a free online trial of distilbart-mnli-12-3, you can try distilbart-mnli-12-3 online for free by clicking the link below.

valhalla distilbart-mnli-12-3 online free url in huggingface.co:

https://huggingface.co/valhalla/distilbart-mnli-12-3

distilbart-mnli-12-3 install

distilbart-mnli-12-3 is an open source model from GitHub that offers a free installation service, and any user can find distilbart-mnli-12-3 on GitHub to install. At the same time, huggingface.co provides the effect of distilbart-mnli-12-3 install, users can directly use distilbart-mnli-12-3 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

distilbart-mnli-12-3 install url in huggingface.co:

https://huggingface.co/valhalla/distilbart-mnli-12-3

Url of distilbart-mnli-12-3

distilbart-mnli-12-3 huggingface.co Url

Provider of distilbart-mnli-12-3 huggingface.co

valhalla
ORGANIZATIONS

Other API from valhalla

huggingface.co

Total runs: 8
Run Growth: 3
Growth Rate: 37.50%
Updated:October 06 2023
huggingface.co

Total runs: 8
Run Growth: 2
Growth Rate: 25.00%
Updated:September 25 2023
huggingface.co

Total runs: 7
Run Growth: 1
Growth Rate: 14.29%
Updated:October 30 2023
huggingface.co

Total runs: 5
Run Growth: 4
Growth Rate: 80.00%
Updated:June 09 2022
huggingface.co

Total runs: 5
Run Growth: 0
Growth Rate: 0.00%
Updated:September 09 2023