Total runs:
1.5M
Run Growth:
-1.1M
Growth Rate:
-72.48%
Sponsored by
APIMart
TinyLlama / TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
Model's Last Updated: December 29 2023
text-generation
Introduction of TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
Model Details of TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
https://github.com/jzhang38/TinyLlama
The TinyLlama project aims to pretrain a 1.1B Llama model on 3 trillion tokens . With some proper optimization, we can achieve this within a span of "just" 90 days using 16 A100-40G GPUs 🚀🚀. The training has started on 2023-09-01.
We adopted exactly the same architecture and tokenizer as Llama 2. This means TinyLlama can be plugged and played in many open-source projects built upon Llama. Besides, TinyLlama is compact with only 1.1B parameters. This compactness allows it to cater to a multitude of applications demanding a restricted computation and memory footprint.
This Collection
This collection contains all checkpoints after the 1T fix. Branch name indicates the step and number of tokens seen.
Eval
| Model | Pretrain Tokens | HellaSwag | Obqa | WinoGrande | ARC_c | ARC_e | boolq | piqa | avg |
|---|---|---|---|---|---|---|---|---|---|
| Pythia-1.0B | 300B | 47.16 | 31.40 | 53.43 | 27.05 | 48.99 | 60.83 | 69.21 | 48.30 |
| TinyLlama-1.1B-intermediate-step-50K-104b | 103B | 43.50 | 29.80 | 53.28 | 24.32 | 44.91 | 59.66 | 67.30 | 46.11 |
| TinyLlama-1.1B-intermediate-step-240k-503b | 503B | 49.56 | 31.40 | 55.80 | 26.54 | 48.32 | 56.91 | 69.42 | 48.28 |
| TinyLlama-1.1B-intermediate-step-480k-1007B | 1007B | 52.54 | 33.40 | 55.96 | 27.82 | 52.36 | 59.54 | 69.91 | 50.22 |
| TinyLlama-1.1B-intermediate-step-715k-1.5T | 1.5T | 53.68 | 35.20 | 58.33 | 29.18 | 51.89 | 59.08 | 71.65 | 51.29 |
| TinyLlama-1.1B-intermediate-step-955k-2T | 2T | 54.63 | 33.40 | 56.83 | 28.07 | 54.67 | 63.21 | 70.67 | 51.64 |
| TinyLlama-1.1B-intermediate-step-1195k-token-2.5T | 2.5T | 58.96 | 34.40 | 58.72 | 31.91 | 56.78 | 63.21 | 73.07 | 53.86 |
Runs of TinyLlama TinyLlama-1.1B-intermediate-step-1195k-token-2.5T on huggingface.co
810
Total runs
0
24-hour runs
-6
3-day runs
12
7-day runs
395
30-day runs
More Information About TinyLlama-1.1B-intermediate-step-1195k-token-2.5T huggingface.co Model
Url of TinyLlama-1.1B-intermediate-step-1195k-token-2.5T
TinyLlama-1.1B-intermediate-step-1195k-token-2.5T huggingface.co Url
Provider of TinyLlama-1.1B-intermediate-step-1195k-token-2.5T huggingface.co
Other API from TinyLlama
Updated:March 17 2024
Total runs:
88.1K
Run Growth:
-23.8K
Growth Rate:
-27.36%
Total runs:
29.3K
Run Growth:
-7.2K
Growth Rate:
-24.54%
Total runs:
10.0K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8.1K
Run Growth:
1.1K
Growth Rate:
13.69%
Total runs:
4.2K
Run Growth:
-305
Growth Rate:
-7.35%
Total runs:
2.8K
Run Growth:
534
Growth Rate:
18.91%
Total runs:
2.7K
Run Growth:
2.7K
Growth Rate:
99.55%
Total runs:
2.5K
Run Growth:
1.0K
Growth Rate:
41.89%
Total runs:
2.3K
Run Growth:
1.4K
Growth Rate:
62.39%
Total runs:
2.0K
Run Growth:
290
Growth Rate:
14.26%
Total runs:
1.9K
Run Growth:
435
Growth Rate:
23.24%
Total runs:
1.7K
Run Growth:
840
Growth Rate:
49.67%
Total runs:
1.5K
Run Growth:
395
Growth Rate:
26.07%
Total runs:
537
Run Growth:
-151
Growth Rate:
-27.36%
Total runs:
396
Run Growth:
-67
Growth Rate:
-16.92%
Total runs:
352
Run Growth:
-85
Growth Rate:
-23.61%
Total runs:
247
Run Growth:
-39
Growth Rate:
-15.18%
Total runs:
143
Run Growth:
12
Growth Rate:
8.39%
Total runs:
113
Run Growth:
-3
Growth Rate:
-2.61%
Total runs:
19
Run Growth:
-10
Growth Rate:
-52.63%
Total runs:
10
Run Growth:
1
Growth Rate:
10.00%
Total runs:
5
Run Growth:
1
Growth Rate:
20.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%