Total runs:
31.5K
Run Growth:
-8.5K
Growth Rate:
-26.87%
Sponsored by
APIMart
FreedomIntelligence / Apollo-MoE-7B
Model's Last Updated: April 07 2025
question-answering
Introduction of Apollo-MoE-7B
Model Details of Apollo-MoE-7B
Democratizing Medical LLMs For Much More Languages
Covering 12 Major Languages including English, Chinese, French, Hindi, Spanish, Arabic, Russian, Japanese, Korean, German, Italian, Portuguese and 38 Minor Languages So far.
📃 Paper • 🌐 Demo • 🤗 ApolloMoEDataset • 🤗 ApolloMoEBench • 🤗 Models •🌐 Apollo • 🌐 ApolloMoE
🌈 Update
- [2024.10.15] ApolloMoE repo is published！🎉
Languages Coverage
12 Major Languages and 38 Minor Languages
Click to view the Languages Coverage
Architecture
Click to view the MoE routing image
Results
Dense
🤗 Apollo2-0.5B • 🤗 Apollo2-1.5B • 🤗 Apollo2-2B
🤗 Apollo2-3.8B • 🤗 Apollo2-7B • 🤗 Apollo2-9B
Click to view the Dense Models Results
Post-MoE
🤗 Apollo-MoE-0.5B • 🤗 Apollo-MoE-1.5B • 🤗 Apollo-MoE-7B
Click to view the Post-MoE Models Results
Usage Format
Apollo2
- 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>
- 2B, 9B: User:{query}\nAssistant:{response}<eos>
- 3.8B: <|user|>\n{query}<|end|><|assisitant|>\n{response}<|end|>
Apollo-MoE
- 0.5B, 1.5B, 7B: User:{query}\nAssistant:{response}<|endoftext|>
Dataset & Evaluation
-
Dataset 🤗 ApolloMoEDataset
-
Evaluation 🤗 ApolloMoEBench
Click to expand
-
EN:
- MedQA-USMLE
- MedMCQA
- PubMedQA : Because the results fluctuated too much, they were not used in the paper.
-
MMLU-Medical
- Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
-
ZH:
- MedQA-MCMLE
-
CMB-single
: Not used in the paper
- Randomly sample 2,000 multiple-choice questions with single answer.
-
CMMLU-Medical
- Anatomy, Clinical_knowledge, College_medicine, Genetics, Nutrition, Traditional_chinese_medicine, Virology
-
CExam
: Not used in the paper
- Randomly sample 2,000 multiple-choice questions
-
ES: Head_qa
-
FR:
- Frenchmedmcqa
-
[MMLU_FR]
- Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
-
HI: MMLU_HI
- Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
-
AR: MMLU_AR
- Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
-
JA: IgakuQA
-
KO: KorMedMCQA
-
IT:
- MedExpQA
-
[MMLU_IT]
- Clinical knowledge, Medical genetics, Anatomy, Professional medicine, College biology, College medicine
-
DE: BioInstructQA : German part
-
PT: BioInstructQA : Portuguese part
-
RU: RuMedBench
-
Results reproduction
Click to expand
We take Apollo2-7B or Apollo-MoE-0.5B as example
-
Download Dataset for project:
bash 0.download_data.sh -
Prepare test and dev data for specific model:
- Create test data for with special token
bash 1.data_process_test&dev.sh -
Prepare train data for specific model (Create tokenized data in advance):
- You can adjust data Training order and Training Epoch in this step
bash 2.data_process_train.sh -
Train the model
- If you want to train in Multi Nodes please refer to ./src/sft/training_config/zero_multi.yaml
bash 3.single_node_train.sh -
Evaluate your model: Generate score for benchmark
bash 4.eval.sh
Citation
Please use the following citation if you intend to use our dataset for training or evaluation:
@misc{zheng2024efficientlydemocratizingmedicalllms,
title={Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Experts},
author={Guorui Zheng and Xidong Wang and Juhao Liang and Nuo Chen and Yuping Zheng and Benyou Wang},
year={2024},
eprint={2410.10626},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2410.10626},
}
Runs of FreedomIntelligence Apollo-MoE-7B on huggingface.co
102
Total runs
4
24-hour runs
13
3-day runs
-88
7-day runs
-63
30-day runs
More Information About Apollo-MoE-7B huggingface.co Model
Url of Apollo-MoE-7B
Apollo-MoE-7B huggingface.co Url
Provider of Apollo-MoE-7B huggingface.co
Other API from FreedomIntelligence
Updated:January 17 2026
Total runs:
2.2K
Run Growth:
764
Growth Rate:
34.66%
Total runs:
1.4K
Run Growth:
800
Growth Rate:
56.62%
Total runs:
1.4K
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1.3K
Run Growth:
-315
Growth Rate:
-24.21%
Total runs:
859
Run Growth:
80
Growth Rate:
9.31%
Total runs:
839
Run Growth:
82
Growth Rate:
9.77%
Total runs:
667
Run Growth:
461
Growth Rate:
69.74%
Total runs:
643
Run Growth:
410
Growth Rate:
63.76%
Total runs:
601
Run Growth:
0
Growth Rate:
0.00%
Total runs:
557
Run Growth:
-365
Growth Rate:
-65.53%
Total runs:
500
Run Growth:
-180
Growth Rate:
-36.00%
Total runs:
458
Run Growth:
148
Growth Rate:
30.33%
Total runs:
375
Run Growth:
4
Growth Rate:
1.07%
Total runs:
318
Run Growth:
291
Growth Rate:
93.87%
Total runs:
314
Run Growth:
202
Growth Rate:
64.33%
Total runs:
308
Run Growth:
-23
Growth Rate:
-7.47%
Total runs:
298
Run Growth:
-161
Growth Rate:
-54.03%
Total runs:
283
Run Growth:
28
Growth Rate:
9.76%
Total runs:
274
Run Growth:
126
Growth Rate:
45.99%
Total runs:
249
Run Growth:
-62
Growth Rate:
-25.41%
Total runs:
232
Run Growth:
-105
Growth Rate:
-45.26%
Total runs:
225
Run Growth:
-119
Growth Rate:
-53.36%
Total runs:
208
Run Growth:
177
Growth Rate:
85.10%
Total runs:
202
Run Growth:
161
Growth Rate:
79.70%
Total runs:
173
Run Growth:
86
Growth Rate:
51.50%
Total runs:
168
Run Growth:
43
Growth Rate:
26.22%
Total runs:
161
Run Growth:
114
Growth Rate:
70.81%
Total runs:
159
Run Growth:
101
Growth Rate:
64.74%
Total runs:
158
Run Growth:
16
Growth Rate:
10.19%
Total runs:
151
Run Growth:
110
Growth Rate:
91.67%
Total runs:
144
Run Growth:
-2
Growth Rate:
-1.39%
Total runs:
143
Run Growth:
108
Growth Rate:
77.14%
Total runs:
132
Run Growth:
45
Growth Rate:
33.58%
Total runs:
130
Run Growth:
114
Growth Rate:
95.00%
Total runs:
129
Run Growth:
4
Growth Rate:
3.10%
Total runs:
122
Run Growth:
-74
Growth Rate:
-60.66%
Total runs:
116
Run Growth:
-7
Growth Rate:
-6.03%
Total runs:
115
Run Growth:
0
Growth Rate:
0.00%
Total runs:
111
Run Growth:
0
Growth Rate:
0.00%
Total runs:
105
Run Growth:
-137
Growth Rate:
-130.48%
Total runs:
103
Run Growth:
1
Growth Rate:
0.97%
Total runs:
99
Run Growth:
54
Growth Rate:
54.55%
Total runs:
98
Run Growth:
-126
Growth Rate:
-123.53%
Total runs:
98
Run Growth:
-220
Growth Rate:
-222.22%
Total runs:
91
Run Growth:
32
Growth Rate:
35.16%
Total runs:
89
Run Growth:
73
Growth Rate:
82.02%
Total runs:
77
Run Growth:
68
Growth Rate:
88.31%
Total runs:
75
Run Growth:
-23
Growth Rate:
-30.67%
Total runs:
74
Run Growth:
0
Growth Rate:
0.00%
Total runs:
72
Run Growth:
-11
Growth Rate:
-15.49%
Total runs:
71
Run Growth:
52
Growth Rate:
75.36%
Total runs:
67
Run Growth:
-105
Growth Rate:
-156.72%
Total runs:
66
Run Growth:
0
Growth Rate:
0.00%
Total runs:
66
Run Growth:
6
Growth Rate:
8.70%
Total runs:
65
Run Growth:
24
Growth Rate:
36.92%
Total runs:
64
Run Growth:
7
Growth Rate:
10.94%
Total runs:
61
Run Growth:
-67
Growth Rate:
-109.84%
Total runs:
59
Run Growth:
26
Growth Rate:
44.83%
Total runs:
57
Run Growth:
13
Growth Rate:
22.03%
Total runs:
56
Run Growth:
43
Growth Rate:
76.79%
Total runs:
53
Run Growth:
0
Growth Rate:
0.00%
Total runs:
50
Run Growth:
-858
Growth Rate:
-1716.00%
Total runs:
50
Run Growth:
2
Growth Rate:
4.00%
Total runs:
47
Run Growth:
-85
Growth Rate:
-180.85%
Total runs:
45
Run Growth:
17
Growth Rate:
38.64%
Total runs:
44
Run Growth:
44
Growth Rate:
100.00%
Total runs:
43
Run Growth:
17
Growth Rate:
37.78%
Total runs:
41
Run Growth:
-38
Growth Rate:
-97.44%
Total runs:
40
Run Growth:
21
Growth Rate:
51.22%
Total runs:
40
Run Growth:
-2
Growth Rate:
-4.88%
Total runs:
40
Run Growth:
7
Growth Rate:
16.67%
Total runs:
37
Run Growth:
-3
Growth Rate:
-8.11%
Total runs:
36
Run Growth:
16
Growth Rate:
44.44%
Total runs:
36
Run Growth:
10
Growth Rate:
27.78%
Total runs:
35
Run Growth:
5
Growth Rate:
14.29%
Total runs:
35
Run Growth:
0
Growth Rate:
0.00%
Total runs:
35
Run Growth:
19
Growth Rate:
54.29%
Total runs:
33
Run Growth:
21
Growth Rate:
63.64%
Total runs:
32
Run Growth:
-16
Growth Rate:
-51.61%
Total runs:
31
Run Growth:
5
Growth Rate:
16.13%
Total runs:
30
Run Growth:
25
Growth Rate:
83.33%
Total runs:
26
Run Growth:
0
Growth Rate:
0.00%
Total runs:
25
Run Growth:
12
Growth Rate:
52.17%
Total runs:
24
Run Growth:
14
Growth Rate:
60.87%
Total runs:
24
Run Growth:
5
Growth Rate:
21.74%
Total runs:
24
Run Growth:
9
Growth Rate:
32.14%
Total runs:
24
Run Growth:
16
Growth Rate:
66.67%
Total runs:
23
Run Growth:
9
Growth Rate:
47.37%
Total runs:
22
Run Growth:
12
Growth Rate:
54.55%
Total runs:
22
Run Growth:
10
Growth Rate:
45.45%
Total runs:
22
Run Growth:
12
Growth Rate:
54.55%
Total runs:
22
Run Growth:
10
Growth Rate:
45.45%
Total runs:
21
Run Growth:
0
Growth Rate:
0.00%
Total runs:
21
Run Growth:
4
Growth Rate:
20.00%
Total runs:
20
Run Growth:
3
Growth Rate:
18.75%
Total runs:
18
Run Growth:
4
Growth Rate:
22.22%
Total runs:
16
Run Growth:
-2
Growth Rate:
-13.33%
Total runs:
15
Run Growth:
4
Growth Rate:
26.67%
Total runs:
15
Run Growth:
9
Growth Rate:
60.00%