Total runs:
1.6K
Run Growth:
1.3K
Growth Rate:
83.73%
Sponsored by
APIMart
FlameF0X / TinyMoE-200m-2x16
Model's Last Updated: August 07 2026
text-generation
Introduction of TinyMoE-200m-2x16
Model Details of TinyMoE-200m-2x16
TinyMoE-100M-2x16
TinyMoE-100M-2x16 is a compact, highly efficient Sparse Mixture of Experts (MoE) language model built upon the Mixtral/Mistral architecture. Designed for research, edge applications, and resource-constrained environments, this model leverages an expert-routing mechanism to balance a larger total parameter capacity with ultra-low computational overhead during inference.
Model Details
- Architecture: Sparse Mixture of Experts (MoE)
- Total Parameters: ~200m total parameters
- Active Parameters per Token: ~50M active parameters
-
Expert Configuration:
16 total local experts, 2 active experts routed per token (
num_experts_per_tok": 2) - Context Length: 1024 tokens
- Base Architecture: Mixtral / Mistral For Causal LM
- License: MIT
Training Data
This model was trained on
FineWeb-Edu-Dedup
60% and
Cosmopedia-v2
40%.
Quick Start
You can load and experiment with this model using the Hugging Face
transformers
library:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "FlameF0X/TinyMoE-100m-2x16"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
input_text = "Wikipedia is a free"
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=50)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Runs of FlameF0X TinyMoE-200m-2x16 on huggingface.co
928
Total runs
0
24-hour runs
0
3-day runs
532
7-day runs
928
30-day runs
More Information About TinyMoE-200m-2x16 huggingface.co Model
Url of TinyMoE-200m-2x16
TinyMoE-200m-2x16 huggingface.co Url
Provider of TinyMoE-200m-2x16 huggingface.co
Other API from FlameF0X
Updated:July 05 2026
Total runs:
1.0K
Run Growth:
138
Growth Rate:
13.50%
Total runs:
620
Run Growth:
0
Growth Rate:
0.00%
Total runs:
446
Run Growth:
-658
Growth Rate:
-147.53%
Total runs:
386
Run Growth:
97
Growth Rate:
25.13%
Total runs:
306
Run Growth:
306
Growth Rate:
100.00%
Total runs:
278
Run Growth:
41
Growth Rate:
14.75%
Total runs:
228
Run Growth:
65
Growth Rate:
28.51%
Total runs:
212
Run Growth:
212
Growth Rate:
100.00%
Total runs:
182
Run Growth:
88
Growth Rate:
48.35%
Total runs:
178
Run Growth:
0
Growth Rate:
0.00%
Total runs:
131
Run Growth:
0
Growth Rate:
0.00%
Total runs:
87
Run Growth:
-62
Growth Rate:
-71.26%
Total runs:
80
Run Growth:
44
Growth Rate:
55.00%
Total runs:
67
Run Growth:
49
Growth Rate:
73.13%
Total runs:
61
Run Growth:
10
Growth Rate:
15.38%
Total runs:
50
Run Growth:
32
Growth Rate:
64.00%
Total runs:
49
Run Growth:
-41
Growth Rate:
-70.69%
Total runs:
27
Run Growth:
3
Growth Rate:
11.11%
Total runs:
20
Run Growth:
6
Growth Rate:
30.00%
Total runs:
19
Run Growth:
-3
Growth Rate:
-13.64%
Total runs:
16
Run Growth:
16
Growth Rate:
100.00%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
10
Growth Rate:
100.00%
Total runs:
7
Run Growth:
7
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
1
Growth Rate:
100.00%
Total runs:
1
Run Growth:
-30
Growth Rate:
-3000.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%