BamiBERT: A New BERT-based Language Model for Vietnamese
We introduce BamiBERT, a new BERT-based pre-trained language model for Vietnamese that addresses key limitations of PhoBERT -- the current de facto Vietnamese text encoder. Trained from scratch on a 129GB corpus of general-domain Vietnamese text for 20 epochs, BamiBERT supports an extended context length of up to 2048 tokens and operates directly on raw input, eliminating the need for external word segmentation. Across 8 Vietnamese benchmarks, it achieves the best score on 11 of 15 metrics and the second-best on 3 others, setting a new state of the art among "base"-sized Vietnamese encoders and demonstrating strong cross-domain generalization.
The general architecture and experimental results of BamiBERT can be found in our
paper
:
@article{BamiBERT,
title = {{BamiBERT: A New BERT-based Language Model for Vietnamese}},
author = {Dat Quoc Nguyen and Thinh Pham and Chi Tran and Linh The Nguyen},
journal = {arXiv preprint},
volume = {arXiv:2607.02259},
year = {2026}
}
Please CITE
our paper when BamiBERT is used to help produce published results or is incorporated into other software.
Please find fine-tuning examples for various downstream tasks, such as token classification (e.g., NER), text classification and question answering, using
transformers
.
The model is intended for research and educational purposes.
Limitations and Bias
BamiBERT is not designed for fluent text generation like GPT-style models. It may also suffer from temporal concept drift: although its pre-training corpus (cutoff December 2022) is more recent than those of other general-domain models (pre-2021), it may not reflect current language use. BamiBERT may also be biased toward standard Northern Vietnamese and underperform on Central and Southern dialects.
Runs of Qualcomm-AI-Research BamiBERT on huggingface.co
28.7K
Total runs
1.1K
24-hour runs
19
3-day runs
-172
7-day runs
18.3K
30-day runs
More Information About BamiBERT huggingface.co Model
BamiBERT huggingface.co is an AI model on huggingface.co that provides BamiBERT's model effect (), which can be used instantly with this Qualcomm-AI-Research BamiBERT model. huggingface.co supports a free trial of the BamiBERT model, and also provides paid use of the BamiBERT. Support call BamiBERT model through api, including Node.js, Python, http.
BamiBERT huggingface.co is an online trial and call api platform, which integrates BamiBERT's modeling effects, including api services, and provides a free online trial of BamiBERT, you can try BamiBERT online for free by clicking the link below.
Qualcomm-AI-Research BamiBERT online free url in huggingface.co:
BamiBERT is an open source model from GitHub that offers a free installation service, and any user can find BamiBERT on GitHub to install. At the same time, huggingface.co provides the effect of BamiBERT install, users can directly use BamiBERT installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
