Sponsored by
APIMart
maywell / Synatra-7B-v0.3-RP
Model's Last Updated: November 21 2023
text-generation
Introduction of Synatra-7B-v0.3-RP
Model Details of Synatra-7B-v0.3-RP
Synatra-7B-v0.3-RP๐ง
Support Me
์๋ํธ๋ผ๋ ๊ฐ์ธ ํ๋ก์ ํธ๋ก, 1์ธ์ ์์์ผ๋ก ๊ฐ๋ฐ๋๊ณ ์์ต๋๋ค. ๋ชจ๋ธ์ด ๋ง์์ ๋์
จ๋ค๋ฉด ์ฝ๊ฐ์ ์ฐ๊ตฌ๋น ์ง์์ ์ด๋จ๊น์?
Wanna be a sponser? Contact me on Telegram AlzarTakkarsen
License
This model is strictly non-commercial ( cc-by-nc-4.0 ) use only. The "Model" is completely free (ie. base model, derivates, merges/mixes) to use for non-commercial purposes as long as the the included cc-by-nc-4.0 license in any parent repository, and the non-commercial use statute remains, regardless of other models' licences. The licence can be changed after new model released. If you are to use this model for commercial purpose, Contact me.
Model Details
Base Model
mistralai/Mistral-7B-Instruct-v0.1
Trained On
A6000 48GB * 8
Instruction format
It follows ChatML format.
TODO
-
RP ๊ธฐ๋ฐ ํ๋ ๋ชจ๋ธ ์ ์ -
๋ฐ์ดํฐ์ ์ ์ - ์ธ์ด ์ดํด๋ฅ๋ ฅ ๊ฐ์
-
์์ ๋ณด์ - ํ ํฌ๋์ด์ ๋ณ๊ฒฝ
Model Benchmark
Ko-LLM-Leaderboard
On Benchmarking...
Implementation Code
Since, chat_template already contains insturction format above. You can use the code below.
from transformers import AutoModelForCausalLM, AutoTokenizer
device = "cuda" # the device to load the model onto
model = AutoModelForCausalLM.from_pretrained("maywell/Synatra-7B-v0.3-RP")
tokenizer = AutoTokenizer.from_pretrained("maywell/Synatra-7B-v0.3-RP")
messages = [
{"role": "user", "content": "๋ฐ๋๋๋ ์๋ ํ์์์ด์ผ?"},
]
encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
model_inputs = encodeds.to(device)
model.to(device)
generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
decoded = tokenizer.batch_decode(generated_ids)
print(decoded[0])
Why It's benchmark score is lower than preview version?
Apparently , Preview model uses Alpaca Style prompt which has no pre-fix. But ChatML do.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 57.38 |
| ARC (25-shot) | 62.2 |
| HellaSwag (10-shot) | 82.29 |
| MMLU (5-shot) | 60.8 |
| TruthfulQA (0-shot) | 52.64 |
| Winogrande (5-shot) | 76.48 |
| GSM8K (5-shot) | 21.15 |
| DROP (3-shot) | 46.06 |
Runs of maywell Synatra-7B-v0.3-RP on huggingface.co
8.1K
Total runs
0
24-hour runs
-20
3-day runs
-46
7-day runs
70
30-day runs
More Information About Synatra-7B-v0.3-RP huggingface.co Model
Url of Synatra-7B-v0.3-RP
Synatra-7B-v0.3-RP huggingface.co Url
Provider of Synatra-7B-v0.3-RP huggingface.co
Other API from maywell
Total runs:
905
Run Growth:
-25
Growth Rate:
-2.76%
Total runs:
897
Run Growth:
-45
Growth Rate:
-5.02%
Total runs:
893
Run Growth:
-48
Growth Rate:
-5.38%
Total runs:
832
Run Growth:
-34
Growth Rate:
-4.09%
Total runs:
826
Run Growth:
-28
Growth Rate:
-3.39%
Total runs:
825
Run Growth:
-42
Growth Rate:
-5.09%
Total runs:
430
Run Growth:
-542
Growth Rate:
-126.34%
Total runs:
185
Run Growth:
0
Growth Rate:
0.00%
Total runs:
183
Run Growth:
115
Growth Rate:
63.19%
Total runs:
133
Run Growth:
17
Growth Rate:
12.78%
Total runs:
129
Run Growth:
29
Growth Rate:
22.14%
Total runs:
120
Run Growth:
-352
Growth Rate:
-293.33%
Total runs:
100
Run Growth:
-306
Growth Rate:
-306.00%
Total runs:
98
Run Growth:
-309
Growth Rate:
-315.31%
Total runs:
86
Run Growth:
53
Growth Rate:
60.23%
Total runs:
78
Run Growth:
62
Growth Rate:
79.49%
Total runs:
78
Run Growth:
-4
Growth Rate:
-5.13%
Total runs:
71
Run Growth:
-12
Growth Rate:
-16.90%
Total runs:
61
Run Growth:
9
Growth Rate:
14.75%
Total runs:
55
Run Growth:
-7
Growth Rate:
-12.73%
Total runs:
37
Run Growth:
-5
Growth Rate:
-13.51%
Total runs:
25
Run Growth:
14
Growth Rate:
60.87%
Total runs:
24
Run Growth:
-2
Growth Rate:
-8.33%
Total runs:
24
Run Growth:
7
Growth Rate:
36.84%
Total runs:
18
Run Growth:
2
Growth Rate:
12.50%
Total runs:
16
Run Growth:
1
Growth Rate:
6.25%
Total runs:
16
Run Growth:
10
Growth Rate:
62.50%
Total runs:
14
Run Growth:
9
Growth Rate:
64.29%
Total runs:
14
Run Growth:
2
Growth Rate:
14.29%
Total runs:
13
Run Growth:
11
Growth Rate:
84.62%
Total runs:
12
Run Growth:
-30
Growth Rate:
-250.00%
Total runs:
12
Run Growth:
8
Growth Rate:
66.67%
Total runs:
11
Run Growth:
7
Growth Rate:
63.64%
Total runs:
11
Run Growth:
-17
Growth Rate:
-154.55%
Total runs:
11
Run Growth:
7
Growth Rate:
63.64%
Total runs:
10
Run Growth:
3
Growth Rate:
30.00%
Total runs:
10
Run Growth:
6
Growth Rate:
60.00%
Total runs:
10
Run Growth:
9
Growth Rate:
90.00%
Total runs:
9
Run Growth:
-5
Growth Rate:
-55.56%
Total runs:
9
Run Growth:
5
Growth Rate:
55.56%
Total runs:
9
Run Growth:
-1
Growth Rate:
-11.11%
Total runs:
7
Run Growth:
2
Growth Rate:
28.57%
Total runs:
6
Run Growth:
2
Growth Rate:
33.33%
Total runs:
6
Run Growth:
-3
Growth Rate:
-50.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%