Sponsored by YouArt
Toolify Toolify logo
AI Models
Yi-Ko-34B-Instruct

maywell / Yi-Ko-34B-Instruct

Run on huggingface.co
huggingface.co
Total runs: 21
24-hour runs: 0
7-day runs: 0
30-day runs: 9
Model's Last Updated: May 18 2024
text-generation

Introduction of Yi-Ko-34B-Instruct

Model Details of Yi-Ko-34B-Instruct

Yi Ko 34B Instruct

Training Process
  1. Further trained with Korean corpus.
  2. SFT
  3. DPO (Dataset URL)
Model Info
Context Length Parameter Prompt Template KMMLU(5-shot)
4k(4096) 34B ChatML 49.03
Acknowledgement

The training is supported by Sionic AI .

Original Model Card by beomi

Yi-Ko series models serve as advanced iterations of 01-ai/Yi models, benefiting from an expanded vocabulary and the inclusion of Korean/English corpus in its further pretraining. Just like its predecessor, Yi-Ko series models operate within the broad range of generative text models that stretch from 6 billion to 34 billion parameters. This repository focuses on the 34B pretrained version, which is tailored to fit the Hugging Face Transformers format. For access to the other models, feel free to consult the index provided below.

Model Details

Model Developers Junbum Lee (Beomi)

Variations Yi-Ko-34B will come in a range of parameter sizes — 6B and 34B — with Ko(Korean+English).

Input Models input text only.

Output Models generate text only.

Model Architecture

Yi-Ko series models are an auto-regressive language model that uses an optimized transformer architecture based on Llama-2*.

*Yi model architecture is based on Llama2, so it can be loaded via LlamaForCausalLM class on HF.

Model Name Training Data Params Context Length GQA Trained Tokens LR Train tokens (per batch)
Yi-Ko-34B A mix of Korean + English online data 34B 4k O 40B+ 5e -5 4M

Vocab Expansion

Model Name Vocabulary Size Description
Original Yi-Series 64000 Sentencepiece BPE
Expanded Yi-Ko Series 78464 Sentencepiece BPE. Added Korean vocab and merges

Tokenizing "안녕하세요, 오늘은 날씨가 좋네요.ㅎㅎ"

Model # of tokens Tokens
Original Yi-Series 47 ['<0xEC>', '<0x95>', '<0x88>', '<0xEB>', '<0x85>', '<0x95>', '하', '<0xEC>', '<0x84>', '<0xB8>', '<0xEC>', '<0x9A>', '<0x94>', ',', '▁', '<0xEC>', '<0x98>', '<0xA4>', '<0xEB>', '<0x8A>', '<0x98>', '은', '▁', '<0xEB>', '<0x82>', '<0xA0>', '<0xEC>', '<0x94>', '<0xA8>', '가', '▁', '<0xEC>', '<0xA2>', '<0x8B>', '<0xEB>', '<0x84>', '<0xA4>', '<0xEC>', '<0x9A>', '<0x94>', '.', '<0xE3>', '<0x85>', '<0x8E>', '<0xE3>', '<0x85>', '<0x8E>']
Expanded Yi-Ko Series 10 ['▁안녕', '하세요', ',', '▁오늘은', '▁날', '씨가', '▁좋네요', '.', 'ㅎ', 'ㅎ']
*Equal Korean vocab with Llama-2-Ko Series

Tokenizing "Llama 2: Open Foundation and Fine-Tuned Chat Models"

Model # of tokens Tokens
Original Yi-Series 21 ['The', '▁Y', 'i', '▁series', '▁models', '▁are', '▁large', '▁language', '▁models', '▁trained', '▁from', '▁scratch', '▁by', '▁developers', '▁at', '▁', '0', '1', '.', 'AI', '.']
Expanded Yi-Ko Series 21 ['▁The', '▁Y', 'i', '▁series', '▁models', '▁are', '▁large', '▁language', '▁models', '▁trained', '▁from', '▁scratch', '▁by', '▁developers', '▁at', '▁', '0', '1', '.', 'AI', '.']
*Equal Korean vocab with Llama-2-Ko Series *Since Expanded Yi-Ko Series prepends _ at the beginning of the text(to ensure same tokenization for Korean sentences), it shows negilible difference for the first token on English tokenization.

Model Benchmark

LM Eval Harness - Korean Benchmarks
Tasks Version Filter n-shot Metric Value Stderr
kmmlu_direct N/A none 5 exact_match 0.5027 ± 0.1019
kobest_boolq 1 none 5 acc 0.9202 ± 0.0072
none 5 f1 0.9202 ± N/A
kobest_copa 1 none 5 acc 0.8480 ± 0.0114
none 5 f1 0.8479 ± N/A
kobest_hellaswag 1 none 5 acc 0.5320 ± 0.0223
none 5 f1 0.5281 ± N/A
none 5 acc_norm 0.6340 ± 0.0216
kobest_sentineg 1 none 5 acc 0.9874 ± 0.0056
none 5 f1 0.9874 ± N/A
haerae N/A none 5 acc 0.7965 ± 0.0116
none 5 acc_norm 0.7965 ± 0.0116
- haerae_general_knowledge 1 none 5 acc 0.5114 ± 0.0378
none 5 acc_norm 0.5114 ± 0.0378
- haerae_history 1 none 5 acc 0.8511 ± 0.0260
none 5 acc_norm 0.8511 ± 0.0260
- haerae_loan_word 1 none 5 acc 0.8402 ± 0.0283
none 5 acc_norm 0.8402 ± 0.0283
- haerae_rare_word 1 none 5 acc 0.8642 ± 0.0170
none 5 acc_norm 0.8642 ± 0.0170
- haerae_standard_nomenclature 1 none 5 acc 0.8301 ± 0.0305
none 5 acc_norm 0.8301 ± 0.0305
LICENSE

Follows Yi License

Citation
Acknowledgement

The training is supported by TPU Research Cloud program.

Runs of maywell Yi-Ko-34B-Instruct on huggingface.co

21
Total runs
0
24-hour runs
1
3-day runs
0
7-day runs
9
30-day runs

More Information About Yi-Ko-34B-Instruct huggingface.co Model

More Yi-Ko-34B-Instruct license Visit here:

https://choosealicense.com/licenses/yi-license

Yi-Ko-34B-Instruct huggingface.co

Yi-Ko-34B-Instruct huggingface.co is an AI model on huggingface.co that provides Yi-Ko-34B-Instruct's model effect (), which can be used instantly with this maywell Yi-Ko-34B-Instruct model. huggingface.co supports a free trial of the Yi-Ko-34B-Instruct model, and also provides paid use of the Yi-Ko-34B-Instruct. Support call Yi-Ko-34B-Instruct model through api, including Node.js, Python, http.

Yi-Ko-34B-Instruct huggingface.co Url

https://huggingface.co/maywell/Yi-Ko-34B-Instruct

maywell Yi-Ko-34B-Instruct online free

Yi-Ko-34B-Instruct huggingface.co is an online trial and call api platform, which integrates Yi-Ko-34B-Instruct's modeling effects, including api services, and provides a free online trial of Yi-Ko-34B-Instruct, you can try Yi-Ko-34B-Instruct online for free by clicking the link below.

maywell Yi-Ko-34B-Instruct online free url in huggingface.co:

https://huggingface.co/maywell/Yi-Ko-34B-Instruct

Yi-Ko-34B-Instruct install

Yi-Ko-34B-Instruct is an open source model from GitHub that offers a free installation service, and any user can find Yi-Ko-34B-Instruct on GitHub to install. At the same time, huggingface.co provides the effect of Yi-Ko-34B-Instruct install, users can directly use Yi-Ko-34B-Instruct installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Yi-Ko-34B-Instruct install url in huggingface.co:

https://huggingface.co/maywell/Yi-Ko-34B-Instruct

Url of Yi-Ko-34B-Instruct

Yi-Ko-34B-Instruct huggingface.co Url

Provider of Yi-Ko-34B-Instruct huggingface.co

maywell
ORGANIZATIONS
huggingface.co

Other API from maywell

 huggingface.co
maywell/TinyWand-SFT

Total runs: 127
Run Growth: -376
Growth Rate: -296.06%
Updated:January 07 2024
huggingface.co
maywell/PiVoT-SUS-RP

Total runs: 122
Run Growth: -349
Growth Rate: -286.07%
Updated:January 15 2024
huggingface.co
maywell/kiqu-70b

Total runs: 115
Run Growth: -14
Growth Rate: -12.17%
Updated:February 19 2024
huggingface.co
maywell/PiVoT-MoE

Total runs: 112
Run Growth: -353
Growth Rate: -315.18%
Updated:December 17 2023
huggingface.co
maywell/TinyWand-DPO

Total runs: 109
Run Growth: 19
Growth Rate: 17.59%
Updated:February 02 2024
huggingface.co
maywell/RM-Gemma-2B

Total runs: 27
Run Growth: 0
Growth Rate: 0.00%
Updated:January 26 2025
huggingface.co
maywell/l3-211m

Total runs: 22
Run Growth: 14
Growth Rate: 63.64%
Updated:April 30 2024
huggingface.co
maywell/phi_ko1ep

Total runs: 15
Run Growth: 10
Growth Rate: 66.67%
Updated:November 15 2023
huggingface.co
maywell/ko_ocgn_ep1

Total runs: 9
Run Growth: 5
Growth Rate: 55.56%
Updated:November 13 2023
huggingface.co
maywell/o1-reward-g

Total runs: 8
Run Growth: 0
Growth Rate: 0.00%
Updated:December 14 2024
huggingface.co
maywell/160_merged

Total runs: 7
Run Growth: 2
Growth Rate: 28.57%
Updated:December 02 2023
huggingface.co
maywell/Qwen2-0.5B

Total runs: 5
Run Growth: 0
Growth Rate: 0.00%
Updated:July 31 2024