mrm8488 / deberta-v3-small-finetuned-cola

huggingface.co
Total runs: 160
24-hour runs: 0
7-day runs: 62
30-day runs: 128
Model's Last Updated: March 21 2023
text-classification

Introduction of deberta-v3-small-finetuned-cola

Model Details of deberta-v3-small-finetuned-cola

DeBERTa-v3-small fine-tuned on CoLA

This model is a fine-tuned version of microsoft/deberta-v3-small on the GLUE COLA dataset. It achieves the following results on the evaluation set:

  • Loss: 0.4051
  • Matthews Correlation: 0.6333
Model description

DeBERTa improves the BERT and RoBERTa models using disentangled attention and enhanced mask decoder. With those two improvements, DeBERTa out perform RoBERTa on a majority of NLU tasks with 80GB training data.

Please check the official repository for more details and updates.

In DeBERTa V3 , we replaced the MLM objective with the RTD(Replaced Token Detection) objective introduced by ELECTRA for pre-training, as well as some innovations to be introduced in our upcoming paper. Compared to DeBERTa-V2, our V3 version significantly improves the model performance in downstream tasks. You can find a simple introduction about the model from the appendix A11 in our original paper , but we will provide more details in a separate write-up.

The DeBERTa V3 small model comes with 6 layers and a hidden size of 768. Its total parameter number is 143M since we use a vocabulary containing 128K tokens which introduce 98M parameters in the Embedding layer. This model was trained using the 160GB data as DeBERTa V2.

Intended uses & limitations

More information needed

Training and evaluation data

The Corpus of Linguistic Acceptability (CoLA) in its full form consists of 10657 sentences from 23 linguistics publications, expertly annotated for acceptability (grammaticality) by their original authors. The public version provided here contains 9594 sentences belonging to training and development sets, and excludes 1063 sentences belonging to a held out test set.

Training procedure
Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-05
  • train_batch_size: 16
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5.0
Training results
Training Loss Epoch Step Validation Loss Matthews Correlation
No log 1.0 535 0.4051 0.6333
0.3371 2.0 1070 0.4455 0.6531
0.3371 3.0 1605 0.5755 0.6499
0.1305 4.0 2140 0.7188 0.6553
0.1305 5.0 2675 0.8047 0.6700
Framework versions
  • Transformers 4.13.0.dev0
  • Pytorch 1.10.0+cu111
  • Datasets 1.16.1
  • Tokenizers 0.10.3

Runs of mrm8488 deberta-v3-small-finetuned-cola on huggingface.co

160
Total runs
0
24-hour runs
-1
3-day runs
62
7-day runs
128
30-day runs

More Information About deberta-v3-small-finetuned-cola huggingface.co Model

More deberta-v3-small-finetuned-cola license Visit here:

https://choosealicense.com/licenses/mit

deberta-v3-small-finetuned-cola huggingface.co

deberta-v3-small-finetuned-cola huggingface.co is an AI model on huggingface.co that provides deberta-v3-small-finetuned-cola's model effect (), which can be used instantly with this mrm8488 deberta-v3-small-finetuned-cola model. huggingface.co supports a free trial of the deberta-v3-small-finetuned-cola model, and also provides paid use of the deberta-v3-small-finetuned-cola. Support call deberta-v3-small-finetuned-cola model through api, including Node.js, Python, http.

deberta-v3-small-finetuned-cola huggingface.co Url

https://huggingface.co/mrm8488/deberta-v3-small-finetuned-cola

mrm8488 deberta-v3-small-finetuned-cola online free

deberta-v3-small-finetuned-cola huggingface.co is an online trial and call api platform, which integrates deberta-v3-small-finetuned-cola's modeling effects, including api services, and provides a free online trial of deberta-v3-small-finetuned-cola, you can try deberta-v3-small-finetuned-cola online for free by clicking the link below.

mrm8488 deberta-v3-small-finetuned-cola online free url in huggingface.co:

https://huggingface.co/mrm8488/deberta-v3-small-finetuned-cola

deberta-v3-small-finetuned-cola install

deberta-v3-small-finetuned-cola is an open source model from GitHub that offers a free installation service, and any user can find deberta-v3-small-finetuned-cola on GitHub to install. At the same time, huggingface.co provides the effect of deberta-v3-small-finetuned-cola install, users can directly use deberta-v3-small-finetuned-cola installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

deberta-v3-small-finetuned-cola install url in huggingface.co:

https://huggingface.co/mrm8488/deberta-v3-small-finetuned-cola

Url of deberta-v3-small-finetuned-cola

deberta-v3-small-finetuned-cola huggingface.co Url

Provider of deberta-v3-small-finetuned-cola huggingface.co

mrm8488
ORGANIZATIONS

Other API from mrm8488

huggingface.co

Total runs: 855
Run Growth: 264
Growth Rate: 30.88%
Updated:February 23 2024
huggingface.co

Total runs: 213
Run Growth: 64
Growth Rate: 31.68%
Updated:March 19 2023
huggingface.co

Total runs: 72
Run Growth: 32
Growth Rate: 44.44%
Updated:January 24 2024
huggingface.co

Total runs: 60
Run Growth: 46
Growth Rate: 82.14%
Updated:June 20 2023