Introduction of bert-fast-dhivehi-tokenizer-extended
Model Details of bert-fast-dhivehi-tokenizer-extended
BERT Tokenizer Extended for Dhivehi
This is a
BertTokenizerFast
built by extending the original
bert-base-multilingual-cased
with a large Dhivehi corpus. It retains full compatibility with English and other languages while adding wordpiece-level support for Dhivehi script.
How to Use
from transformers import BertTokenizerFast
tokenizer = BertTokenizerFast.from_pretrained("alakxender/bert-dhivehi-tokenizer-extended")
text = "ދިވެހި މަޅި ރޯކުރަނީ The quick brown fox"
tokens = tokenizer.tokenize(text)
print(tokens)
bert-fast-dhivehi-tokenizer-extended huggingface.co is an AI model on huggingface.co that provides bert-fast-dhivehi-tokenizer-extended's model effect (), which can be used instantly with this alakxender bert-fast-dhivehi-tokenizer-extended model. huggingface.co supports a free trial of the bert-fast-dhivehi-tokenizer-extended model, and also provides paid use of the bert-fast-dhivehi-tokenizer-extended. Support call bert-fast-dhivehi-tokenizer-extended model through api, including Node.js, Python, http.
bert-fast-dhivehi-tokenizer-extended huggingface.co is an online trial and call api platform, which integrates bert-fast-dhivehi-tokenizer-extended's modeling effects, including api services, and provides a free online trial of bert-fast-dhivehi-tokenizer-extended, you can try bert-fast-dhivehi-tokenizer-extended online for free by clicking the link below.
alakxender bert-fast-dhivehi-tokenizer-extended online free url in huggingface.co:
bert-fast-dhivehi-tokenizer-extended is an open source model from GitHub that offers a free installation service, and any user can find bert-fast-dhivehi-tokenizer-extended on GitHub to install. At the same time, huggingface.co provides the effect of bert-fast-dhivehi-tokenizer-extended install, users can directly use bert-fast-dhivehi-tokenizer-extended installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
bert-fast-dhivehi-tokenizer-extended install url in huggingface.co: