rasyosef / bert-amharic-tokenizer-24k

huggingface.co
Total runs: 0
24-hour runs: 0
7-day runs: 0
30-day runs: 0
Model's Last Updated: May 11 2024

Introduction of bert-amharic-tokenizer-24k

Model Details of bert-amharic-tokenizer-24k

Amharic WordPiece Tokenizer

This repo contains a WordPiece tokenizer trained on the Amharic subset of the oscar and mc4 datasets. It's the same as the BERT tokenizer but trained from scratch on an amharic text dataset, with a vocabulary size of 24576 .

How to use

You can load the tokenizer from huggingface hub as follows.

from transformers import AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("rasyosef/bert-amharic-tokenizer")
tokenizer.tokenize("የዓለምአቀፉ ነጻ ንግድ መስፋፋት ድህነትን ለማሸነፍ በሚደረገው ትግል አንዱ ጠቃሚ መሣሪያ ሊሆን መቻሉ ብዙ የሚነገርለት ጉዳይ ነው።")

Output:

['የዓለም', '##አ', '##ቀፉ', 'ነጻ', 'ንግድ', 'መስፋፋት', 'ድህነትን', 'ለማሸነፍ', 'በሚደረገው', 'ትግል', 'አንዱ', 'ጠቃሚ', 'መሣሪያ', 'ሊሆን', 'መቻሉ', 'ብዙ', 'የሚነገር', '##ለት', 'ጉዳይ', 'ነው', '።']

Runs of rasyosef bert-amharic-tokenizer-24k on huggingface.co

0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs

More Information About bert-amharic-tokenizer-24k huggingface.co Model

More bert-amharic-tokenizer-24k license Visit here:

https://choosealicense.com/licenses/mit

bert-amharic-tokenizer-24k huggingface.co

bert-amharic-tokenizer-24k huggingface.co is an AI model on huggingface.co that provides bert-amharic-tokenizer-24k's model effect (), which can be used instantly with this rasyosef bert-amharic-tokenizer-24k model. huggingface.co supports a free trial of the bert-amharic-tokenizer-24k model, and also provides paid use of the bert-amharic-tokenizer-24k. Support call bert-amharic-tokenizer-24k model through api, including Node.js, Python, http.

bert-amharic-tokenizer-24k huggingface.co Url

https://huggingface.co/rasyosef/bert-amharic-tokenizer-24k

rasyosef bert-amharic-tokenizer-24k online free

bert-amharic-tokenizer-24k huggingface.co is an online trial and call api platform, which integrates bert-amharic-tokenizer-24k's modeling effects, including api services, and provides a free online trial of bert-amharic-tokenizer-24k, you can try bert-amharic-tokenizer-24k online for free by clicking the link below.

rasyosef bert-amharic-tokenizer-24k online free url in huggingface.co:

https://huggingface.co/rasyosef/bert-amharic-tokenizer-24k

bert-amharic-tokenizer-24k install

bert-amharic-tokenizer-24k is an open source model from GitHub that offers a free installation service, and any user can find bert-amharic-tokenizer-24k on GitHub to install. At the same time, huggingface.co provides the effect of bert-amharic-tokenizer-24k install, users can directly use bert-amharic-tokenizer-24k installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

bert-amharic-tokenizer-24k install url in huggingface.co:

https://huggingface.co/rasyosef/bert-amharic-tokenizer-24k

Url of bert-amharic-tokenizer-24k

bert-amharic-tokenizer-24k huggingface.co Url

Provider of bert-amharic-tokenizer-24k huggingface.co

rasyosef
ORGANIZATIONS

Other API from rasyosef

huggingface.co

Total runs: 5.1K
Run Growth: 794
Growth Rate: 15.69%
Updated:October 07 2025
huggingface.co

Total runs: 659
Run Growth: 598
Growth Rate: 90.74%
Updated:October 07 2025
huggingface.co

Total runs: 7
Run Growth: -1
Growth Rate: -14.29%
Updated:September 16 2024