Sponsored by
APIMart
davidkim205 / iris-7b
Model's Last Updated: April 04 2024
translation
Introduction of iris-7b
Model Details of iris-7b
iris
Iris is a model for Korean-English sentence translation based on deep learning.
It is used to translate Korean sentences into English or English sentences into Korean by utilizing advanced natural language processing technology. The model is trained to understand the grammar, vocabulary, and context of each language and generate appropriate translations. Iris provides efficient and accurate translation and can be used in a variety of applications.
Model Details
- Model Developers : davidkim(changyeon kim)
- Repository : will be updated soon.
- base mode : mistralai/Mistral-7B-v0.2
- dataset : translation_v3_346k
usage
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch
repo = "davidkim205/iris-7b"
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype=torch.bfloat16, device_map='auto')
tokenizer = AutoTokenizer.from_pretrained(repo)
def generate(prompt):
encoding = tokenizer(
prompt,
return_tensors='pt',
return_token_type_ids=False
).to("cuda")
gen_tokens = model.generate(
**encoding,
max_new_tokens=2048,
temperature=1.0,
num_beams=5,
)
prompt_end_size = encoding.input_ids.shape[1]
result = tokenizer.decode(gen_tokens[0, prompt_end_size:])
return result
def translate_ko2en(text):
prompt = f"[INST] 다음 문장을 영어로 번역하세요.{text} [/INST]"
return generate(prompt)
def translate_en2ko(text):
prompt = f"[INST] 다음 문장을 한글로 번역하세요.{text} [/INST]"
return generate(prompt)
def main():
while True:
text = input('>')
en_text = translate_ko2en(text)
ko_text = translate_en2ko(en_text)
print('en_text', en_text)
print('ko_text', ko_text)
if __name__ == "__main__":
main()
output
$ python iris_test.py
Downloading shards: 100%|█████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 4.72it/s]
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████| 3/3 [00:02<00:00, 1.07it/s]
>아이리스는 딥러닝을 기반으로 한 한-영어 문장 번역을 위한 모델이다.
en_text Iris is a model for Korean-to-English sentence translation based on deep learning.</s>
ko_text 아이리스는 딥러닝을 기반으로 한 한국어-영어 문장 번역을 위한 모델이다.</s>
template
ko -> en
[INST] 다음 문장을 영어로 번역하세요.{text} [/INST]
en -> ko
"[INST] 다음 문장을 한글로 번역하세요.{text} [/INST]"
dataset info : translation_v3_346k
The dataset is not made public due to licensing issues.
| src | ratio | description |
|---|---|---|
| aihub-MTPE | 5.56% | 기계번역 품질 사후검증 데이터셋 |
| aihub-techsci2 | 5.56% | ICT, 전기/전자 등 기술과학 분야 한영 번역 데이터셋 |
| aihub-expertise | 5.56% | 의료, 금융, 스포츠 등 전문분야 한영 번역 데이터셋 |
| aihub-humanities | 5.56% | 인문학 분야 한영 번역 데이터셋 |
| sharegpt-deepl-ko-translation | 5.56% | shareGPT 데이터셋을 질답 형식에서 한영 번역 형식으로 변환한 데이터셋 |
| aihub-MT-new-corpus | 5.56% | 기계 번역 앱 구축용 한영 번역 데이터셋 |
| aihub-socialsci | 5.56% | 법률, 교육, 경제 등 사회과학 분야 한영 번역 데이터셋 |
| korean-parallel-corpora | 5.56% | 한영 번역 병렬 데이터셋 |
| aihub-parallel-translation | 5.56% | 발화 유형 및 분야별 한영 번역 데이터셋 |
| aihub-food | 5.56% | 식품 분야 영한 번역 데이터셋 |
| aihub-techsci | 5.56% | ICT, 전기/전자 등 기술과학 분야 한영 번역 데이터셋 |
| para_pat | 5.56% | ParaPat 데이터셋의 영어-한국어 subset |
| aihub-speechtype-based-machine-translation | 5.56% | 발화 유형별 영한 번역 데이터셋 |
| koopus100 | 5.56% | OPUS-100 데이터셋의 영어-한국어 subset |
| aihub-basicsci | 5.56% | 수학, 물리학 등 기초과학 분야 한영 번역 데이터셋 |
| aihub-broadcast-content | 5.56% | 방송 콘텐츠 분야 한영 번역 데이터셋 |
| aihub-patent | 5.56% | 특허명세서 영한 번역 데이터셋 |
| aihub-colloquial | 5.56% | 신조어, 약어 등을 포함하는 구어체 한영 번역 데이터셋 |
Please refer to the url below for information on aihub licensing.
https://aihub.or.kr/partcptnmlrd/inqry/view.do?currMenu=144&topMenu=104
Evaluation
https://github.com/davidkim205/translation
| TYPE | Model | BLEU | SBLEU | Duplicate | Length Exceeds |
|---|---|---|---|---|---|
| HuggingFace | facebook/nllb-200-distilled-1.3B | 0.26 | 0.30 | 1 | 3 |
| HuggingFace | jbochi/madlad400-10b-mt | 0.29 | 0.38 | 3 | 6 |
| HuggingFace | Unbabel/TowerInstruct-7B-v0.1 | 0.32 | 0.39 | 1 | 9 |
| HuggingFace | squarelike/Gugugo-koen-7B-V1.1 | 0.32 | 0.36 | 1 | 3 |
| HuggingFace | maywell/Synatra-7B-v0.3-Translation | 0.35 | 0.41 | 1 | 2 |
| Cloud | deepl | 0.39 | 0.45 | 0 | 1 |
| Cloud | azure | 0.40 | 0.49 | 0 | 3 |
| Cloud | 0.40 | 0.49 | 0 | 2 | |
| Cloud | papago | 0.43 | 0.51 | 0 | 3 |
| HuggingFace | davidkim205/iris-7b ( ours ) | 0.40 | 0.43 | 0 | 3 |
Runs of davidkim205 iris-7b on huggingface.co
91
Total runs
0
24-hour runs
-1
3-day runs
3
7-day runs
49
30-day runs
More Information About iris-7b huggingface.co Model
Url of iris-7b
iris-7b huggingface.co Url
Provider of iris-7b huggingface.co
Other API from davidkim205
Total runs:
388
Run Growth:
214
Growth Rate:
55.15%
Total runs:
141
Run Growth:
31
Growth Rate:
21.99%
Total runs:
131
Run Growth:
-120
Growth Rate:
-91.60%
Total runs:
44
Run Growth:
-127
Growth Rate:
-288.64%
Total runs:
41
Run Growth:
15
Growth Rate:
36.59%
Total runs:
31
Run Growth:
0
Growth Rate:
0.00%
Total runs:
25
Run Growth:
12
Growth Rate:
48.00%
Total runs:
24
Run Growth:
7
Growth Rate:
29.17%
Total runs:
23
Run Growth:
12
Growth Rate:
52.17%
Total runs:
23
Run Growth:
13
Growth Rate:
54.17%
Total runs:
21
Run Growth:
7
Growth Rate:
33.33%
Total runs:
21
Run Growth:
8
Growth Rate:
38.10%
Total runs:
20
Run Growth:
15
Growth Rate:
75.00%
Total runs:
20
Run Growth:
13
Growth Rate:
61.90%
Total runs:
19
Run Growth:
16
Growth Rate:
84.21%
Total runs:
17
Run Growth:
4
Growth Rate:
23.53%
Total runs:
13
Run Growth:
3
Growth Rate:
23.08%
Total runs:
12
Run Growth:
1
Growth Rate:
8.33%
Total runs:
11
Run Growth:
8
Growth Rate:
72.73%
Total runs:
8
Run Growth:
7
Growth Rate:
87.50%
Total runs:
8
Run Growth:
6
Growth Rate:
75.00%
Total runs:
8
Run Growth:
2
Growth Rate:
25.00%
Total runs:
8
Run Growth:
3
Growth Rate:
37.50%
Total runs:
6
Run Growth:
3
Growth Rate:
50.00%
Total runs:
6
Run Growth:
1
Growth Rate:
16.67%
Total runs:
5
Run Growth:
-134
Growth Rate:
-6700.00%
Total runs:
2
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%