deliciouscat / deberta-v3-base-encoder-decoder-v0.2

huggingface.co
Total runs: 16
24-hour runs: 0
7-day runs: 1
30-day runs: 9
Model's Last Updated: May 19 2024
text-generation

Introduction of deberta-v3-base-encoder-decoder-v0.2

Model Details of deberta-v3-base-encoder-decoder-v0.2

Encoder-Decoder model with DeBERTa encoder

pre-trained models
  • Encoder: microsoft/deberta-v3-small

  • Decoder: deliciouscat/deberta-v3-base-decoder-v0.1 (6 transformer layers, 8 attention heads)

-> 297511524(298M) params

Data used

HuggingFaceFW/fineweb -> sampled 124800

Training hparams
  • optimizer: AdamW, lr=2.3e-5, betas=(0.875, 0.997)

  • batch size: 12 (maximal on Colab pro A100 env)

-> training on denoising objective (BART)

How to use
from transformers import AutoTokenizer, EncoderDecoderModel

model = EncoderDecoderModel.from_pretrained("deliciouscat/deberta-v3-base-encoder-decoder-v0.2")
tokenizer = AutoTokenizer.from_pretrained("deliciouscat/deberta-v3-base-encoder-decoder-v0.2")
Future work!
  • train more scientific data

  • fine-tune on keyword extraction task

Runs of deliciouscat deberta-v3-base-encoder-decoder-v0.2 on huggingface.co

16
Total runs
0
24-hour runs
1
3-day runs
1
7-day runs
9
30-day runs

More Information About deberta-v3-base-encoder-decoder-v0.2 huggingface.co Model

deberta-v3-base-encoder-decoder-v0.2 huggingface.co

deberta-v3-base-encoder-decoder-v0.2 huggingface.co is an AI model on huggingface.co that provides deberta-v3-base-encoder-decoder-v0.2's model effect (), which can be used instantly with this deliciouscat deberta-v3-base-encoder-decoder-v0.2 model. huggingface.co supports a free trial of the deberta-v3-base-encoder-decoder-v0.2 model, and also provides paid use of the deberta-v3-base-encoder-decoder-v0.2. Support call deberta-v3-base-encoder-decoder-v0.2 model through api, including Node.js, Python, http.

deberta-v3-base-encoder-decoder-v0.2 huggingface.co Url

https://huggingface.co/deliciouscat/deberta-v3-base-encoder-decoder-v0.2

deliciouscat deberta-v3-base-encoder-decoder-v0.2 online free

deberta-v3-base-encoder-decoder-v0.2 huggingface.co is an online trial and call api platform, which integrates deberta-v3-base-encoder-decoder-v0.2's modeling effects, including api services, and provides a free online trial of deberta-v3-base-encoder-decoder-v0.2, you can try deberta-v3-base-encoder-decoder-v0.2 online for free by clicking the link below.

deliciouscat deberta-v3-base-encoder-decoder-v0.2 online free url in huggingface.co:

https://huggingface.co/deliciouscat/deberta-v3-base-encoder-decoder-v0.2

deberta-v3-base-encoder-decoder-v0.2 install

deberta-v3-base-encoder-decoder-v0.2 is an open source model from GitHub that offers a free installation service, and any user can find deberta-v3-base-encoder-decoder-v0.2 on GitHub to install. At the same time, huggingface.co provides the effect of deberta-v3-base-encoder-decoder-v0.2 install, users can directly use deberta-v3-base-encoder-decoder-v0.2 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

deberta-v3-base-encoder-decoder-v0.2 install url in huggingface.co:

https://huggingface.co/deliciouscat/deberta-v3-base-encoder-decoder-v0.2

Url of deberta-v3-base-encoder-decoder-v0.2

deberta-v3-base-encoder-decoder-v0.2 huggingface.co Url

Provider of deberta-v3-base-encoder-decoder-v0.2 huggingface.co

deliciouscat
ORGANIZATIONS

Other API from deliciouscat