MoritzLaurer / ModernBERT-base-zeroshot-v2.0

huggingface.co
Total runs: 283
24-hour runs: 0
7-day runs: 83
30-day runs: 83
Model's Last Updated: January 24 2025
text-classification

Introduction of ModernBERT-base-zeroshot-v2.0

Model Details of ModernBERT-base-zeroshot-v2.0

ModernBERT-base-zeroshot-v2.0

Model description

This model is answerdotai/ModernBERT-base fine-tuned on the same dataset mix as the zeroshot-v2.0 models in the Zeroshot Classifiers Collection .

General takeaways:
  • The model is very fast and memory efficient. It's multiple times faster and consumes multiple times less memory than DeBERTav3. The memory efficiency enables larger batch sizes. I got a ~2x speed increase by enabling bf16 (instead of fp16).
  • It performs slightly worse then DeBERTav3 on average on the tasks tested below.
  • I'm in the process of preparing a newer version trained on better synthetic data to make full use of the 8k context window and to update the training mix of the older zeroshot-v2.0 models.
Training results

Per-dataset breakdown:

Datasets Mean Mean w/o NLI mnli_m mnli_mm fevernli anli_r1 anli_r2 anli_r3 wanli lingnli wellformedquery rottentomatoes amazonpolarity imdb yelpreviews hatexplain massive banking77 emotiondair emocontext empathetic agnews yahootopics biasframes_sex biasframes_offensive biasframes_intent financialphrasebank appreviews hateoffensive trueteacher spam wikitoxic_toxicaggregated wikitoxic_obscene wikitoxic_identityhate wikitoxic_threat wikitoxic_insult manifesto capsotu
Accuracy 0.831 0.835 0.932 0.936 0.884 0.763 0.647 0.657 0.823 0.889 0.753 0.864 0.949 0.935 0.974 0.798 0.788 0.727 0.789 0.793 0.489 0.893 0.717 0.927 0.851 0.859 0.907 0.952 0.926 0.726 0.978 0.912 0.914 0.93 0.951 0.906 0.476 0.708
F1 macro 0.813 0.818 0.925 0.93 0.872 0.74 0.61 0.611 0.81 0.874 0.751 0.864 0.949 0.935 0.974 0.751 0.738 0.746 0.733 0.798 0.475 0.893 0.712 0.919 0.851 0.859 0.892 0.952 0.847 0.721 0.966 0.912 0.914 0.93 0.942 0.906 0.329 0.637
Inference text/sec (A100 40GB GPU, batch=128) 3472.0 3474.0 2338.0 4416.0 2993.0 2959.0 2904.0 3003.0 4647.0 4486.0 5032.0 4354.0 2466.0 1140.0 1582.0 4392.0 5446.0 5296.0 4904.0 4787.0 2251.0 4042.0 1884.0 4048.0 4032.0 4121.0 4275.0 3746.0 4485.0 1114.0 4322.0 2260.0 2274.0 2189.0 2085.0 2410.0 3933.0 4388.0
Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 32
  • eval_batch_size: 128
  • seed: 42
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_ratio: 0.06
  • num_epochs: 2
Framework versions
  • Transformers 4.48.0.dev0
  • Pytorch 2.5.1+cu124
  • Datasets 3.2.0
  • Tokenizers 0.21.0

Runs of MoritzLaurer ModernBERT-base-zeroshot-v2.0 on huggingface.co

283
Total runs
0
24-hour runs
5
3-day runs
83
7-day runs
83
30-day runs

More Information About ModernBERT-base-zeroshot-v2.0 huggingface.co Model

More ModernBERT-base-zeroshot-v2.0 license Visit here:

https://choosealicense.com/licenses/apache-2.0

ModernBERT-base-zeroshot-v2.0 huggingface.co

ModernBERT-base-zeroshot-v2.0 huggingface.co is an AI model on huggingface.co that provides ModernBERT-base-zeroshot-v2.0's model effect (), which can be used instantly with this MoritzLaurer ModernBERT-base-zeroshot-v2.0 model. huggingface.co supports a free trial of the ModernBERT-base-zeroshot-v2.0 model, and also provides paid use of the ModernBERT-base-zeroshot-v2.0. Support call ModernBERT-base-zeroshot-v2.0 model through api, including Node.js, Python, http.

ModernBERT-base-zeroshot-v2.0 huggingface.co Url

https://huggingface.co/MoritzLaurer/ModernBERT-base-zeroshot-v2.0

MoritzLaurer ModernBERT-base-zeroshot-v2.0 online free

ModernBERT-base-zeroshot-v2.0 huggingface.co is an online trial and call api platform, which integrates ModernBERT-base-zeroshot-v2.0's modeling effects, including api services, and provides a free online trial of ModernBERT-base-zeroshot-v2.0, you can try ModernBERT-base-zeroshot-v2.0 online for free by clicking the link below.

MoritzLaurer ModernBERT-base-zeroshot-v2.0 online free url in huggingface.co:

https://huggingface.co/MoritzLaurer/ModernBERT-base-zeroshot-v2.0

ModernBERT-base-zeroshot-v2.0 install

ModernBERT-base-zeroshot-v2.0 is an open source model from GitHub that offers a free installation service, and any user can find ModernBERT-base-zeroshot-v2.0 on GitHub to install. At the same time, huggingface.co provides the effect of ModernBERT-base-zeroshot-v2.0 install, users can directly use ModernBERT-base-zeroshot-v2.0 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

ModernBERT-base-zeroshot-v2.0 install url in huggingface.co:

https://huggingface.co/MoritzLaurer/ModernBERT-base-zeroshot-v2.0

Url of ModernBERT-base-zeroshot-v2.0

ModernBERT-base-zeroshot-v2.0 huggingface.co Url

Provider of ModernBERT-base-zeroshot-v2.0 huggingface.co

MoritzLaurer
ORGANIZATIONS

Other API from MoritzLaurer