Total runs:
2.2M
Run Growth:
-188
Growth Rate:
-0.01%
Sponsored by
APIMart
llm-jp / Dense-btx-code-expert-152M
Model's Last Updated: February 27 2025
Introduction of Dense-btx-code-expert-152M
Model Details of Dense-btx-code-expert-152M
Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization
📄 [Paper] | 🤗 [Hugging Face] 📁 [Dataset] 💻 [Code] | 📊 [Log]
Model Index
We provide model checkpoints for all experiments to ensure reproducibility of the results presented in Tables 1 and 2.
Table 1
| Model | Link |
|---|---|
| 1 Dense 152M | Link |
| 2 MoE FS 8x152M | Link |
| 3 MoE BTX 8x152M | Link |
| 4 MoE NU 8x152M | Link |
| 5 MoE RNU (r=0.5) 8x152M | Link |
| 6 MoE DU (r=0.5) 8x152M | Link |
| 7 MoE DU (r=1.0) 8x152M | Link |
| 8 Dense 1.5B | Link |
| 9 MoE FS 8x1.5B | Link |
| 10 MoE BTX 8x1.5B | Link |
| 11 MoE NU 8x1.5B | Link |
| 12 MoE RNU (r=0.5) 8x1.5B | Link |
| 13 MoE DU (r=0.5) 8x1.5B | Link |
| 14 MoE DU (r=1.0) 8x1.5B | Link |
Table 2
| Model | Link |
|---|---|
| 1 Dense 3.7B | Link |
| 2 MoE FS 8x3.7B | Link |
| 3 MoE DU (r=0.5) 8x3.7B | Link |
| 4 Dense 13B | Link |
| 5 Dense 3.7B | Link |
BTX Experts
| Model | Link |
|---|---|
| Japanese expert 152M | Link |
| English expert 152M | Link |
| Code expert 152M | Link |
| Japanese expert 1.5B | Link |
| English expert 1.5B | Link |
| Code expert 1.5B | Link |
How to cite
If you find our work helpful, please feel free to cite.
@inproceedings{
nakamura2025dropupcycling,
title={Drop-Upcycling: Training Sparse Mixture of Experts with Partial Re-initialization},
author={Taishi Nakamura and Takuya Akiba and Kazuki Fujii and Yusuke Oda and Rio Yokota and Jun Suzuki},
booktitle={The Thirteenth International Conference on Learning Representations},
year={2025},
url={https://openreview.net/forum?id=gx1wHnf5Vp}
}
Runs of llm-jp Dense-btx-code-expert-152M on huggingface.co
92
Total runs
2
24-hour runs
3
3-day runs
11
7-day runs
72
30-day runs
More Information About Dense-btx-code-expert-152M huggingface.co Model
Url of Dense-btx-code-expert-152M
Dense-btx-code-expert-152M huggingface.co Url
Provider of Dense-btx-code-expert-152M huggingface.co
Other API from llm-jp
Updated:September 27 2024
Total runs:
39.5K
Run Growth:
13.2K
Growth Rate:
33.57%
Total runs:
34.4K
Run Growth:
33.7K
Growth Rate:
97.84%
Total runs:
30.7K
Run Growth:
1.1K
Growth Rate:
3.66%
Total runs:
7.2K
Run Growth:
1.3K
Growth Rate:
18.03%
Total runs:
6.5K
Run Growth:
-3.3K
Growth Rate:
-52.05%
Total runs:
5.9K
Run Growth:
5.4K
Growth Rate:
90.64%
Total runs:
5.2K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
4.9K
Run Growth:
2.3K
Growth Rate:
47.90%
Total runs:
4.4K
Run Growth:
-588
Growth Rate:
-13.47%
Total runs:
4.4K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
3.9K
Run Growth:
1.1K
Growth Rate:
28.51%
Total runs:
2.0K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.8K
Run Growth:
111
Growth Rate:
6.55%
Total runs:
1.5K
Run Growth:
-319
Growth Rate:
-21.57%
Total runs:
1.3K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.3K
Run Growth:
-148
Growth Rate:
-11.56%
Total runs:
1.2K
Run Growth:
141
Growth Rate:
11.78%
Total runs:
1.2K
Run Growth:
-33
Growth Rate:
-2.78%
Total runs:
1.1K
Run Growth:
24
Growth Rate:
2.11%
Total runs:
999
Run Growth:
0
Growth Rate:
0.00%
Total runs:
887
Run Growth:
-447
Growth Rate:
-50.39%
Total runs:
772
Run Growth:
-33
Growth Rate:
-4.27%
Total runs:
590
Run Growth:
0
Growth Rate:
0.00%
Total runs:
536
Run Growth:
0
Growth Rate:
0.00%
Total runs:
518
Run Growth:
119
Growth Rate:
22.97%
Total runs:
500
Run Growth:
66
Growth Rate:
13.20%
Total runs:
459
Run Growth:
28
Growth Rate:
6.11%
Total runs:
445
Run Growth:
-13
Growth Rate:
-2.92%
Total runs:
296
Run Growth:
270
Growth Rate:
91.22%
Total runs:
223
Run Growth:
105
Growth Rate:
48.17%
Total runs:
207
Run Growth:
80
Growth Rate:
39.60%
Total runs:
193
Run Growth:
66
Growth Rate:
35.48%
Total runs:
190
Run Growth:
190
Growth Rate:
100.00%
Total runs:
164
Run Growth:
133
Growth Rate:
81.10%
Total runs:
160
Run Growth:
144
Growth Rate:
90.00%
Total runs:
148
Run Growth:
118
Growth Rate:
82.52%
Total runs:
138
Run Growth:
94
Growth Rate:
68.12%
Total runs:
133
Run Growth:
0
Growth Rate:
0.00%
Total runs:
123
Run Growth:
92
Growth Rate:
76.03%
Total runs:
121
Run Growth:
0
Growth Rate:
0.00%
Total runs:
118
Run Growth:
92
Growth Rate:
79.31%
Total runs:
118
Run Growth:
88
Growth Rate:
75.86%
Total runs:
114
Run Growth:
100
Growth Rate:
87.72%
Total runs:
98
Run Growth:
0
Growth Rate:
0.00%
Total runs:
97
Run Growth:
66
Growth Rate:
68.04%
Total runs:
94
Run Growth:
70
Growth Rate:
74.47%
Total runs:
94
Run Growth:
-46
Growth Rate:
-48.94%
Total runs:
93
Run Growth:
77
Growth Rate:
82.80%
Total runs:
88
Run Growth:
76
Growth Rate:
86.36%
Total runs:
87
Run Growth:
67
Growth Rate:
77.01%
Total runs:
71
Run Growth:
0
Growth Rate:
0.00%
Total runs:
54
Run Growth:
0
Growth Rate:
0.00%
Total runs:
40
Run Growth:
22
Growth Rate:
55.00%
Total runs:
38
Run Growth:
0
Growth Rate:
0.00%
Total runs:
23
Run Growth:
12
Growth Rate:
52.17%
Total runs:
14
Run Growth:
4
Growth Rate:
28.57%
Total runs:
14
Run Growth:
13
Growth Rate:
92.86%
Total runs:
14
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
8
Run Growth:
-4
Growth Rate:
-44.44%
Total runs:
8
Run Growth:
-6
Growth Rate:
-75.00%
Total runs:
8
Run Growth:
-1
Growth Rate:
-12.50%
Total runs:
5
Run Growth:
4
Growth Rate:
80.00%
Total runs:
3
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%