Total runs:
734.4K
Run Growth:
603.3K
Growth Rate:
80.92%
Sponsored by
AdsCreator.com
AtomicChat / Ling-3.1-flash-GGUF
Model's Last Updated: October 15 2026
text-generation
Introduction of Ling-3.1-flash-GGUF
Model Details of Ling-3.1-flash-GGUF
How to Run Ling 3.1 flash Locally
GGUF builds of inclusionAI's Ling 3.1 flash with our own importance matrix. The calibration corpora behind our builds are public.
- Runs in llama.cpp build b11571 or newer.
- Day-one upload: more files and the measurements (KL divergence and top-1 against the BF16 model) land on this card today.
Pick a file
| File | Size on disk | KL divergence | top-1 | Status |
|---|---|---|---|---|
AD-Q8_0
|
600.8 GB | 0.00503 | 97.88% | available |
AD-Q6_K
|
478.0 GB | 0.00557 | 97.72% | available |
AD-Q5_K_M
|
400.2 GB | 0.00766 | 97.34% | available |
AD-Q4_K_M
|
354.4 GB | 0.01114 | 96.86% | available |
AD-IQ4_XS
|
337.8 GB | 0.01212 | 96.64% | available |
AD-IQ4_XXS
|
311.4 GB | 0.01391 | 96.46% | available |
AD-IQ3_M
|
277.1 GB | 0.02564 | 95.27% | available |
AD-IQ3_XXS
|
256.4 GB | 0.03436 | 94.48% | available |
AD-IQ2_M
|
218.1 GB | 0.05927 | 92.72% | available |
AD-IQ2_XS
|
198.2 GB | measuring | measuring | uploading today |
AD-IQ2_XXS
|
174.1 GB | 0.11133 | 90.10% | available |
AD-IQ1_M
|
160.9 GB | 0.16597 | 87.86% | available, experimental |
AD-IQ1_S
|
141.8 GB | 0.20448 | 86.41% | available, experimental |
KL divergence
is how far the quantized model's predictions drift from the
BF16 model's. Lower is better, and zero means identical.
top-1
is how often it
picks the same next token the BF16 model would have picked. Both are measured
on held-out text; how is at the bottom. Rows that say "measuring" fill in today.
AD-
marks an
Atomic Dynamic
layout: the routed experts (97% of the weights)
take the low-bit type, while attention, the shared expert, the router and the
dense layers stay at q8_0 or higher. The file name is the type of the experts'
gate and up projections; their down projections, and all experts of the last
five blocks, get a step more.
How much memory it needs
The file has to fit, plus a little for the context. Context is cheap on this model: 63 of its 72 layers use linear attention with a fixed-size state, and only 9 keep a compressed attention cache. That is about 0.5 GB per conversation plus about 10 KB per token.
| Total memory (VRAM or unified) | File |
|---|---|
| 192 GB |
AD-IQ1_M
|
| 256 GB |
AD-IQ2_M
|
| 384 GB |
AD-Q4_K_M
|
| 512 GB |
AD-Q5_K_M
|
| 640 GB |
AD-Q8_0
|
On a Mac, raise the GPU memory limit first, for example
sudo sysctl iogpu.wired_limit_mb=240000
on a 256 GB machine.
Running it
llama-server -hf AtomicChat/Ling-3.1-flash-GGUF:AD-Q4_K_M -ngl 99 -c 32768 --jinja
Each file sits in a folder of its name, split into shards of up to 45 GB; point llama.cpp at the first shard and it loads the rest:
hf download AtomicChat/Ling-3.1-flash-GGUF --include "AD-Q4_K_M/*" --local-dir .
llama-server -m AD-Q4_K_M/Ling-3.1-flash-AD-Q4_K_M-00001-of-00009.gguf -ngl 99 -c 32768 --jinja
The chat template is inside every file. Thinking is on by default; put
detailed thinking off
in the system prompt to turn it off.
How we measured
Reference: BF16 weights converted from inclusionAI's FP8 release and run unquantized (perplexity 2.3590 on the held-out set). Converting FP8 to BF16 rounds each weight once, far below what Q8_0 adds.
Held-out text:
eval_neutral
from our
calibration dataset
,
never used for calibration, 144 chunks of 4096 tokens. Metric: per-token KL
divergence of each file's predictions against the reference, plus top-1
agreement. Hardware: 8x A100-SXM4-80GB, llama.cpp 097cf33d9 for every file.
The importance matrix is the one we computed on 5.5M tokens of the
ling3-moe
corpus (thinking on and off, tool calls, 30 languages, code, long
documents up to 32k tokens), rendered through this model's own chat template.
Model details
Mixture of experts: 512 routed experts in each of 68 layers (the first 4 layers are dense), 8 routed plus 1 shared expert per token, about 24B parameters active of 564B. Hybrid attention: 63 Kimi Delta Attention (linear) layers and 9 multi-head latent attention layers. Hidden size 4096, vocabulary 157,184, context 262,144. The multi-token-prediction head of the release is not included.
Built from inclusionAI's FP8 release (revision
b426279f
), converted to BF16
and quantized with llama.cpp b11571 (097cf33d9); needs llama.cpp with
bailingmoe3
support.
License and credits
MIT, as the original. Base model Ling 3.1 flash by inclusionAI . Quantization and evaluation by AtomicChat .
Runs of AtomicChat Ling-3.1-flash-GGUF on huggingface.co
524
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About Ling-3.1-flash-GGUF huggingface.co Model
Url of Ling-3.1-flash-GGUF
Ling-3.1-flash-GGUF huggingface.co Url
Provider of Ling-3.1-flash-GGUF huggingface.co
Other API from AtomicChat
Updated:October 14 2026
Total runs:
296.3K
Run Growth:
251.0K
Growth Rate:
97.17%
Total runs:
234.6K
Run Growth:
-98.8K
Growth Rate:
-42.09%
Total runs:
119.4K
Run Growth:
102.8K
Growth Rate:
94.36%
Total runs:
53.5K
Run Growth:
-16.9K
Growth Rate:
-30.35%
Total runs:
53.0K
Run Growth:
-17.2K
Growth Rate:
-31.22%
Total runs:
52.9K
Run Growth:
-17.1K
Growth Rate:
-31.06%
Total runs:
48.2K
Run Growth:
-213.8K
Growth Rate:
-459.90%
Total runs:
45.9K
Run Growth:
-15.6K
Growth Rate:
-34.40%
Total runs:
25.0K
Run Growth:
-66.9K
Growth Rate:
-264.09%
Total runs:
14.0K
Run Growth:
7.4K
Growth Rate:
54.69%
Total runs:
13.8K
Run Growth:
9.4K
Growth Rate:
72.11%
Total runs:
12.5K
Run Growth:
2.8K
Growth Rate:
100.00%
Total runs:
11.1K
Run Growth:
5.9K
Growth Rate:
53.88%
Total runs:
10.0K
Run Growth:
5.7K
Growth Rate:
74.53%
Total runs:
8.2K
Run Growth:
3.4K
Growth Rate:
42.99%
Total runs:
8.1K
Run Growth:
2.0K
Growth Rate:
84.29%
Total runs:
7.9K
Run Growth:
5.5K
Growth Rate:
72.76%
Total runs:
7.5K
Run Growth:
4.7K
Growth Rate:
63.29%
Total runs:
7.2K
Run Growth:
2.7K
Growth Rate:
37.78%
Total runs:
6.3K
Run Growth:
3.6K
Growth Rate:
57.21%
Total runs:
6.1K
Run Growth:
759
Growth Rate:
25.30%
Total runs:
5.6K
Run Growth:
1.2K
Growth Rate:
21.24%
Total runs:
4.7K
Run Growth:
2.7K
Growth Rate:
59.51%
Total runs:
3.6K
Run Growth:
1.4K
Growth Rate:
42.84%
Total runs:
3.5K
Run Growth:
1.5K
Growth Rate:
42.52%
Total runs:
3.4K
Run Growth:
2.1K
Growth Rate:
77.34%
Total runs:
3.2K
Run Growth:
423
Growth Rate:
54.23%
Total runs:
2.5K
Run Growth:
124
Growth Rate:
4.89%
Total runs:
2.1K
Run Growth:
724
Growth Rate:
32.67%
Total runs:
2.0K
Run Growth:
-910
Growth Rate:
-46.52%
Total runs:
1.9K
Run Growth:
1.8K
Growth Rate:
100.00%
Total runs:
1.7K
Run Growth:
657
Growth Rate:
37.65%
Total runs:
1.7K
Run Growth:
142
Growth Rate:
8.33%
Total runs:
1.6K
Run Growth:
105
Growth Rate:
7.61%
Total runs:
1.6K
Run Growth:
1.6K
Growth Rate:
100.00%
Total runs:
1.6K
Run Growth:
514
Growth Rate:
98.66%
Total runs:
1.5K
Run Growth:
-86
Growth Rate:
-5.68%
Total runs:
1.4K
Run Growth:
-3.0K
Growth Rate:
-204.14%
Total runs:
1.4K
Run Growth:
348
Growth Rate:
25.18%
Total runs:
1.3K
Run Growth:
1.1K
Growth Rate:
88.93%
Total runs:
1.2K
Run Growth:
1.2K
Growth Rate:
100.00%
Total runs:
1.1K
Run Growth:
54
Growth Rate:
5.03%
Total runs:
1.0K
Run Growth:
1.0K
Growth Rate:
100.00%
Total runs:
1.0K
Run Growth:
1.0K
Growth Rate:
100.00%
Total runs:
998
Run Growth:
458
Growth Rate:
50.27%
Total runs:
910
Run Growth:
347
Growth Rate:
37.72%
Total runs:
872
Run Growth:
100
Growth Rate:
11.59%
Total runs:
812
Run Growth:
223
Growth Rate:
27.26%
Total runs:
735
Run Growth:
-581
Growth Rate:
-78.09%
Total runs:
732
Run Growth:
-6.1K
Growth Rate:
-829.55%
Total runs:
703
Run Growth:
169
Growth Rate:
24.28%
Total runs:
644
Run Growth:
-131
Growth Rate:
-21.13%
Total runs:
618
Run Growth:
241
Growth Rate:
36.08%
Total runs:
602
Run Growth:
602
Growth Rate:
100.00%
Total runs:
572
Run Growth:
142
Growth Rate:
25.22%
Total runs:
480
Run Growth:
-172
Growth Rate:
-35.76%
Total runs:
455
Run Growth:
437
Growth Rate:
100.00%
Total runs:
442
Run Growth:
-35
Growth Rate:
-7.95%
Total runs:
434
Run Growth:
30
Growth Rate:
7.03%
Total runs:
429
Run Growth:
-426
Growth Rate:
-99.30%
Total runs:
423
Run Growth:
22
Growth Rate:
4.74%
Total runs:
357
Run Growth:
0
Growth Rate:
0.00%
Total runs:
349
Run Growth:
-642
Growth Rate:
-181.36%
Total runs:
325
Run Growth:
15
Growth Rate:
4.70%
Total runs:
295
Run Growth:
-130
Growth Rate:
-44.67%
Total runs:
272
Run Growth:
-734
Growth Rate:
-269.85%
Total runs:
233
Run Growth:
-36
Growth Rate:
-15.79%
Total runs:
231
Run Growth:
-32
Growth Rate:
-14.29%
Total runs:
226
Run Growth:
66
Growth Rate:
28.70%
Total runs:
222
Run Growth:
-298
Growth Rate:
-129.57%
Total runs:
214
Run Growth:
-351
Growth Rate:
-161.75%
Total runs:
209
Run Growth:
36
Growth Rate:
15.65%
Total runs:
193
Run Growth:
-61
Growth Rate:
-30.81%
Total runs:
186
Run Growth:
-211
Growth Rate:
-115.30%
Total runs:
171
Run Growth:
-95
Growth Rate:
-56.21%
Total runs:
167
Run Growth:
3
Growth Rate:
1.82%
Total runs:
165
Run Growth:
-55
Growth Rate:
-32.35%
Total runs:
162
Run Growth:
-21
Growth Rate:
-13.29%
Total runs:
155
Run Growth:
-133
Growth Rate:
-87.50%
Total runs:
150
Run Growth:
-244
Growth Rate:
-153.46%
Total runs:
143
Run Growth:
-13
Growth Rate:
-10.74%
Total runs:
128
Run Growth:
-88
Growth Rate:
-67.18%
Total runs:
123
Run Growth:
-192
Growth Rate:
-169.91%
Total runs:
120
Run Growth:
-26
Growth Rate:
-23.01%
Total runs:
117
Run Growth:
117
Growth Rate:
100.00%
Total runs:
117
Run Growth:
104
Growth Rate:
88.89%
Total runs:
96
Run Growth:
-180
Growth Rate:
-183.67%
Total runs:
94
Run Growth:
-192
Growth Rate:
-202.11%
Total runs:
94
Run Growth:
82
Growth Rate:
87.23%
Total runs:
91
Run Growth:
-17
Growth Rate:
-19.32%
Total runs:
90
Run Growth:
0
Growth Rate:
0.00%
Total runs:
90
Run Growth:
-35
Growth Rate:
-37.23%
Total runs:
88
Run Growth:
-115
Growth Rate:
-121.05%
Total runs:
80
Run Growth:
-1
Growth Rate:
-1.49%
Total runs:
79
Run Growth:
-130
Growth Rate:
-164.56%
Total runs:
75
Run Growth:
-67
Growth Rate:
-94.37%