GPT-2-small model pretrained from scratch on the
Minangkabau portion of WikiMulti.
Dataset
Dataset:
dehanalkautsar/WikiMulti
File:
20260801_mi_wiki.parquet
Training field:
text
Architecture
Standard GPT-2-small architecture:
Layers: 12
Hidden size: 768
Attention heads: 12
Maximum context: 1024
Vocabulary size: 50257
All GPT-2 weights were initialized from scratch.
No pretrained GPT-2 model weights were used.
Tokenizer
A separate Byte-Level BPE tokenizer was trained from scratch
using only the Minangkabau training split.
The original English GPT-2 vocabulary and BPE merge rules
were not used.
Vocabulary size: 50257
Training
GPU: NVIDIA RTX A6000
Precision: FP16
TF32: True
Maximum epochs: 50
Batch size per GPU: 8
Gradient accumulation: 4
Sequence length: 1024
Learning rate: 5e-05
Scheduler: linear
Validation frequency: every 500 optimizer steps
Early stopping patience: 3
Early stopping threshold: 0.0
Random seed: 42
Training was stopped after validation loss failed to improve
for 3 consecutive evaluation rounds,
unless the maximum epoch ceiling was reached first.
Best Model
Best validation loss:
1.6707736253738403
Final best-model validation loss:
1.6707736253738403
Perplexity:
5.316279015045347
Runs of dehanalkautsar gpt2-mi-wikimulti on huggingface.co
664
Total runs
8
24-hour runs
29
3-day runs
76
7-day runs
405
30-day runs
More Information About gpt2-mi-wikimulti huggingface.co Model
gpt2-mi-wikimulti huggingface.co
gpt2-mi-wikimulti huggingface.co is an AI model on huggingface.co that provides gpt2-mi-wikimulti's model effect (), which can be used instantly with this dehanalkautsar gpt2-mi-wikimulti model. huggingface.co supports a free trial of the gpt2-mi-wikimulti model, and also provides paid use of the gpt2-mi-wikimulti. Support call gpt2-mi-wikimulti model through api, including Node.js, Python, http.
gpt2-mi-wikimulti huggingface.co is an online trial and call api platform, which integrates gpt2-mi-wikimulti's modeling effects, including api services, and provides a free online trial of gpt2-mi-wikimulti, you can try gpt2-mi-wikimulti online for free by clicking the link below.
dehanalkautsar gpt2-mi-wikimulti online free url in huggingface.co:
gpt2-mi-wikimulti is an open source model from GitHub that offers a free installation service, and any user can find gpt2-mi-wikimulti on GitHub to install. At the same time, huggingface.co provides the effect of gpt2-mi-wikimulti install, users can directly use gpt2-mi-wikimulti installed effect in huggingface.co for debugging and trial. It also supports api for free installation.