Sponsored by
APIMart
appvoid / palmer-004-turbo
Model's Last Updated: Juli 22 2024
text-generation
Introduction of palmer-004-turbo
Model Details of palmer-004-turbo
palmer turbo
This model has a slightly different architecture and training style:
- The model was followed by a continual pretraining (lm_head + embedding layers were tuned).
- Base model was pretrained on 75k instruction/response pairs and merged.
- Similar architecture than palmer series but smaller in context size (8192)
In short, palmer is now half the size, twice the speed and almost same overall performance with a notable improvement on mmlu and arc challenge instead of winogrande. As of Wed 17 Jul, it beats all models =< 0.5b on hellaswag.
As all palmer models, the model is biased to respond to answers without using any specific prompt, feel free to further fine-tune it for your specific use case.
benchmarks
These are zero-shot evaluations performed on current state-of-the-art language models.
| Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average |
|---|---|---|---|---|---|---|
| smollm-360m | 0.2537 | 0.3626 | 0.5350 | 0.7116 | 0.5659 | 0.4858 |
| tinyllama | 0.2577 | 0.3029 | 0.5935 | 0.7329 | 0.5959 | 0.4966 |
| qwen2-0.5b | 0.4413 | 0.2892 | 0.4905 | 0.6931 | 0.5699 | 0.4968 |
| danube3-500m-chat (current sota) | 0.2554 | 0.3626 | 0.6072 | 0.7432 | 0.6140 | 0.5164 |
| palmer-004-turbo | 0.2736 | 0.3558 | 0.6179 | 0.7367 | 0.6117 | 0.5191 |
| palmer-004 | 0.2661 | 0.3490 | 0.6173 | 0.7481 | 0.6417 | 0.5244 |
thanks to
- h2oai: performant base model provider
- teknium: openhermes dataset provider
- unsloth: tooling for training software
Runs of appvoid palmer-004-turbo on huggingface.co
21
Total runs
0
24-hour runs
1
3-day runs
2
7-day runs
-4
30-day runs
More Information About palmer-004-turbo huggingface.co Model
Url of palmer-004-turbo
palmer-004-turbo huggingface.co Url
Provider of palmer-004-turbo huggingface.co
Other API from appvoid
Total runs:
122
Run Growth:
20
Growth Rate:
16.39%
Total runs:
60
Run Growth:
38
Growth Rate:
63.33%
Total runs:
31
Run Growth:
20
Growth Rate:
64.52%
Total runs:
27
Run Growth:
3
Growth Rate:
11.11%
Total runs:
25
Run Growth:
15
Growth Rate:
60.00%
Total runs:
24
Run Growth:
19
Growth Rate:
79.17%
Total runs:
23
Run Growth:
13
Growth Rate:
56.52%
Total runs:
23
Run Growth:
15
Growth Rate:
65.22%
Total runs:
22
Run Growth:
12
Growth Rate:
54.55%
Total runs:
20
Run Growth:
11
Growth Rate:
55.00%
Total runs:
20
Run Growth:
7
Growth Rate:
35.00%
Total runs:
19
Run Growth:
15
Growth Rate:
78.95%
Total runs:
19
Run Growth:
12
Growth Rate:
63.16%
Total runs:
19
Run Growth:
5
Growth Rate:
26.32%
Total runs:
19
Run Growth:
11
Growth Rate:
57.89%
Total runs:
19
Run Growth:
9
Growth Rate:
47.37%
Total runs:
17
Run Growth:
11
Growth Rate:
64.71%
Total runs:
17
Run Growth:
8
Growth Rate:
47.06%
Total runs:
17
Run Growth:
2
Growth Rate:
11.76%
Total runs:
17
Run Growth:
9
Growth Rate:
52.94%
Total runs:
16
Run Growth:
10
Growth Rate:
62.50%
Total runs:
16
Run Growth:
9
Growth Rate:
56.25%
Total runs:
16
Run Growth:
10
Growth Rate:
62.50%
Total runs:
15
Run Growth:
8
Growth Rate:
53.33%