flax-community / t5-base-dutch

huggingface.co
Total runs: 42
24-hour runs: 5
7-day runs: 9
30-day runs: 32
Model's Last Updated: January 11 2022
text-generation

Introduction of t5-base-dutch

Model Details of t5-base-dutch

t5-base-dutch

Created by Yeb Havinga & Dat Nguyen during the Hugging Face community week , organized by HuggingFace and TPU usage sponsored by Google, for the project Pre-train T5 from scratch in Dutch .

See also the fine-tuned t5-base-dutch-demo model, and the demo application Netherformer 📰 , that are based on this model.

5 jan 2022: Model updated. Evaluation accuracy increased from 0.64 to 0.70.

11 jan 2022: See also yhavinga/t5-v1.1-base-dutch-cased with eval acc 0.78

Model
  • Configuration based on google/t5-base
  • 12 layers, 12 heads
  • Dropout set to 0.1
Dataset

This model was trained on the full configuration of cleaned Dutch mC4 , which is the original mC4, except

  • Documents that contained words from a selection of the Dutch and English List of Dirty Naught Obscene and Otherwise Bad Words are removed
  • Sentences with less than 3 words are removed
  • Sentences with a word of more than 1000 characters are removed
  • Documents with less than 5 sentences are removed
  • Documents with "javascript", "lorum ipsum", "terms of use", "privacy policy", "cookie policy", "uses cookies", "use of cookies", "use cookies", "elementen ontbreken", "deze printversie" are removed.
Tokenization

A SentencePiece tokenizer was trained from scratch on this dataset. The total tokens of the full configuration is 34B

Training

The model was trained on the full mc4_nl_cleaned dataset configuration for 1 epoch, consisting of 34B tokens, for 528 482 steps with a batch size of 128 and took 57 hours. A triangle learning rate schedule was used, with peak learning rate 0.005.

Evaluation
  • Loss: 1.38
  • Accuracy: 0.70

Runs of flax-community t5-base-dutch on huggingface.co

42
Total runs
5
24-hour runs
5
3-day runs
9
7-day runs
32
30-day runs

More Information About t5-base-dutch huggingface.co Model

More t5-base-dutch license Visit here:

https://choosealicense.com/licenses/apache-2.0

t5-base-dutch huggingface.co

t5-base-dutch huggingface.co is an AI model on huggingface.co that provides t5-base-dutch's model effect (), which can be used instantly with this flax-community t5-base-dutch model. huggingface.co supports a free trial of the t5-base-dutch model, and also provides paid use of the t5-base-dutch. Support call t5-base-dutch model through api, including Node.js, Python, http.

flax-community t5-base-dutch online free

t5-base-dutch huggingface.co is an online trial and call api platform, which integrates t5-base-dutch's modeling effects, including api services, and provides a free online trial of t5-base-dutch, you can try t5-base-dutch online for free by clicking the link below.

flax-community t5-base-dutch online free url in huggingface.co:

https://huggingface.co/flax-community/t5-base-dutch

t5-base-dutch install

t5-base-dutch is an open source model from GitHub that offers a free installation service, and any user can find t5-base-dutch on GitHub to install. At the same time, huggingface.co provides the effect of t5-base-dutch install, users can directly use t5-base-dutch installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

t5-base-dutch install url in huggingface.co:

https://huggingface.co/flax-community/t5-base-dutch

Url of t5-base-dutch

Provider of t5-base-dutch huggingface.co

flax-community
ORGANIZATIONS

Other API from flax-community