Sponsored by YouArt
Toolify Toolify logo
AI Models
Yi-Ko-34B

beomi / Yi-Ko-34B

Run on huggingface.co
huggingface.co
Total runs: 34
24-hour runs: 1
7-day runs: 5
30-day runs: 23
Model's Last Updated: July 08 2024
text-generation

Introduction of Yi-Ko-34B

Model Details of Yi-Ko-34B

Update

@2024.07.08: Update LICENSE into Apache 2.0!🎉

beomi/Yi-Ko-34B

Yi-Ko series models serve as advanced iterations of 01-ai/Yi models, benefiting from an expanded vocabulary and the inclusion of Korean/English corpus in its further pretraining. Just like its predecessor, Yi-Ko series models operate within the broad range of generative text models that stretch from 6 billion to 34 billion parameters. This repository focuses on the 34B pretrained version, which is tailored to fit the Hugging Face Transformers format. For access to the other models, feel free to consult the index provided below.

Model Details

Model Developers Junbum Lee (Beomi)

Variations Yi-Ko-34B will come in a range of parameter sizes — 6B and 34B — with Ko(Korean+English).

Input Models input text only.

Output Models generate text only.

Model Architecture

Yi-Ko series models are an auto-regressive language model that uses an optimized transformer architecture based on Llama-2*.

*Yi model architecture is based on Llama2, so it can be loaded via LlamaForCausalLM class on HF.

Model Name Training Data Params Context Length GQA Trained Tokens LR Train tokens (per batch)
Yi-Ko-34B A mix of Korean + English online data 34B 4k O 40B+ 5e -5 4M

Vocab Expansion

Model Name Vocabulary Size Description
Original Yi-Series 64000 Sentencepiece BPE
Expanded Yi-Ko Series 78464 Sentencepiece BPE. Added Korean vocab and merges

Tokenizing "안녕하세요, 오늘은 날씨가 좋네요.ㅎㅎ"

Model # of tokens Tokens
Original Yi-Series 47 ['<0xEC>', '<0x95>', '<0x88>', '<0xEB>', '<0x85>', '<0x95>', '하', '<0xEC>', '<0x84>', '<0xB8>', '<0xEC>', '<0x9A>', '<0x94>', ',', '▁', '<0xEC>', '<0x98>', '<0xA4>', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '<0xEB>', '<0x82>', '<0xA0>', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '<0xEC>', '<0xA2>', '<0x8B>', '<0xEB>', '<0x84>', '<0xA4>', '<0xEC>', '<0x9A>', '<0x94>', '.', '<0xE3>', '<0x85>', '<0x8E>', '<0xE3>', '<0x85>', '<0x8E>']
Expanded Yi-Ko Series 10 ['▁안녕', '하세요', ',', '▁오늘은', '▁날', '씨가', '▁좋네요', '.', 'ㅎ', 'ㅎ']
*Equal Korean vocab with Llama-2-Ko Series

Tokenizing "Llama 2: Open Foundation and Fine-Tuned Chat Models"

Model # of tokens Tokens
Original Yi-Series 21 ['The', '▁Y', 'i', '▁series', '▁models', '▁are', '▁large', '▁language', '▁models', '▁trained', '▁from', '▁scratch', '▁by', '▁developers', '▁at', '▁', '0', '1', '.', 'AI', '.']
Expanded Yi-Ko Series 21 ['▁The', '▁Y', 'i', '▁series', '▁models', '▁are', '▁large', '▁language', '▁models', '▁trained', '▁from', '▁scratch', '▁by', '▁developers', '▁at', '▁', '0', '1', '.', 'AI', '.']
*Equal Korean vocab with Llama-2-Ko Series *Since Expanded Yi-Ko Series prepends _ at the beginning of the text(to ensure same tokenization for Korean sentences), it shows negilible difference for the first token on English tokenization.

Model Benchmark

LM Eval Harness - Korean Benchmarks
Tasks Version Filter n-shot Metric Value Stderr
kmmlu_direct N/A none 5 exact_match 0.5027 ± 0.1019
kobest_boolq 1 none 5 acc 0.9202 ± 0.0072
none 5 f1 0.9202 ± N/A
kobest_copa 1 none 5 acc 0.8480 ± 0.0114
none 5 f1 0.8479 ± N/A
kobest_hellaswag 1 none 5 acc 0.5320 ± 0.0223
none 5 f1 0.5281 ± N/A
none 5 acc_norm 0.6340 ± 0.0216
kobest_sentineg 1 none 5 acc 0.9874 ± 0.0056
none 5 f1 0.9874 ± N/A
haerae N/A none 5 acc 0.7965 ± 0.0116
none 5 acc_norm 0.7965 ± 0.0116
- haerae_general_knowledge 1 none 5 acc 0.5114 ± 0.0378
none 5 acc_norm 0.5114 ± 0.0378
- haerae_history 1 none 5 acc 0.8511 ± 0.0260
none 5 acc_norm 0.8511 ± 0.0260
- haerae_loan_word 1 none 5 acc 0.8402 ± 0.0283
none 5 acc_norm 0.8402 ± 0.0283
- haerae_rare_word 1 none 5 acc 0.8642 ± 0.0170
none 5 acc_norm 0.8642 ± 0.0170
- haerae_standard_nomenclature 1 none 5 acc 0.8301 ± 0.0305
none 5 acc_norm 0.8301 ± 0.0305
LICENSE

Apache 2.0 (for research)

For commercial purpose, mailto: [email protected] to acquire Yi-Ko sereis commercial license.

Acknowledgement

The training is supported by TPU Research Cloud program.

Runs of beomi Yi-Ko-34B on huggingface.co

34
Total runs
1
24-hour runs
1
3-day runs
5
7-day runs
23
30-day runs

More Information About Yi-Ko-34B huggingface.co Model

More Yi-Ko-34B license Visit here:

https://choosealicense.com/licenses/apache-2.0

Yi-Ko-34B huggingface.co

Yi-Ko-34B huggingface.co is an AI model on huggingface.co that provides Yi-Ko-34B's model effect (), which can be used instantly with this beomi Yi-Ko-34B model. huggingface.co supports a free trial of the Yi-Ko-34B model, and also provides paid use of the Yi-Ko-34B. Support call Yi-Ko-34B model through api, including Node.js, Python, http.

Yi-Ko-34B huggingface.co Url

https://huggingface.co/beomi/Yi-Ko-34B

beomi Yi-Ko-34B online free

Yi-Ko-34B huggingface.co is an online trial and call api platform, which integrates Yi-Ko-34B's modeling effects, including api services, and provides a free online trial of Yi-Ko-34B, you can try Yi-Ko-34B online for free by clicking the link below.

beomi Yi-Ko-34B online free url in huggingface.co:

https://huggingface.co/beomi/Yi-Ko-34B

Yi-Ko-34B install

Yi-Ko-34B is an open source model from GitHub that offers a free installation service, and any user can find Yi-Ko-34B on GitHub to install. At the same time, huggingface.co provides the effect of Yi-Ko-34B install, users can directly use Yi-Ko-34B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Yi-Ko-34B install url in huggingface.co:

https://huggingface.co/beomi/Yi-Ko-34B

Url of Yi-Ko-34B

Yi-Ko-34B huggingface.co Url

Provider of Yi-Ko-34B huggingface.co

beomi
ORGANIZATIONS
huggingface.co

Other API from beomi

 huggingface.co
beomi/KcELECTRA-base

Total runs: 6.0K
Run Growth: 3.0K
Growth Rate: 50.07%
Updated:May 23 2024
huggingface.co
beomi/kcbert-base

Total runs: 3.5K
Run Growth: -3.4K
Growth Rate: -97.20%
Updated:March 30 2023
huggingface.co
beomi/llama-2-ko-7b

Total runs: 530
Run Growth: 254
Growth Rate: 47.92%
Updated:December 27 2023
huggingface.co
beomi/Yi-Ko-6B

Total runs: 385
Run Growth: 228
Growth Rate: 59.22%
Updated:July 08 2024
huggingface.co
beomi/kcbert-large

Total runs: 365
Run Growth: 164
Growth Rate: 44.93%
Updated:March 30 2023
huggingface.co
beomi/gemma-ko-2b

Total runs: 209
Run Growth: 129
Growth Rate: 61.72%
Updated:March 26 2024
huggingface.co
beomi/Yi-Ko-DUS-9B

Total runs: 183
Run Growth: 176
Growth Rate: 96.17%
Updated:July 08 2024
huggingface.co
beomi/KoRWKV-6B

Total runs: 152
Run Growth: 76
Growth Rate: 50.00%
Updated:July 20 2023
huggingface.co
beomi/gemma-ko-7b

Total runs: 120
Run Growth: 19
Growth Rate: 15.83%
Updated:March 26 2024
huggingface.co
beomi/kollama-13b

Total runs: 111
Run Growth: 56
Growth Rate: 50.45%
Updated:June 28 2023
huggingface.co
beomi/kollama-7b

Total runs: 109
Run Growth: 40
Growth Rate: 36.70%
Updated:July 11 2023
huggingface.co
beomi/gemma-mling-7b

Total runs: 49
Run Growth: 26
Growth Rate: 53.06%
Updated:August 21 2024
huggingface.co
beomi/KoRWKV-1.5B

Total runs: 33
Run Growth: 21
Growth Rate: 63.64%
Updated:November 13 2023
huggingface.co
beomi/KcT5-dev

Total runs: 32
Run Growth: 28
Growth Rate: 87.50%
Updated:March 01 2022
huggingface.co
beomi/kobert

Total runs: 23
Run Growth: 12
Growth Rate: 52.17%
Updated:September 15 2023
huggingface.co
beomi/kcgpt2

Total runs: 23
Run Growth: 13
Growth Rate: 56.52%
Updated:November 23 2021
huggingface.co
beomi/kcgpt2-dev

Total runs: 11
Run Growth: -5
Growth Rate: -45.45%
Updated:May 21 2021
huggingface.co
beomi/KcBERT-v2023

Total runs: 9
Run Growth: 5
Growth Rate: 55.56%
Updated:May 07 2023
huggingface.co
beomi/exKcBERT-paws

Total runs: 8
Run Growth: 4
Growth Rate: 50.00%
Updated:June 11 2021
huggingface.co
beomi/KcRoBERTa-dev

Total runs: 6
Run Growth: 5
Growth Rate: 83.33%
Updated:March 08 2023
huggingface.co
beomi/kollama-33b

Total runs: 6
Run Growth: 1
Growth Rate: 16.67%
Updated:June 29 2023
huggingface.co
beomi/qwen3-8b-dmax

Total runs: 3
Run Growth: -5
Growth Rate: -166.67%
Updated:May 04 2026
huggingface.co
beomi/KcT5

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:February 10 2022
huggingface.co
beomi/exKcBERT

Total runs: 0
Run Growth: 0
Growth Rate: 0.00%
Updated:June 10 2021