Total runs:
22.9K
Run Growth:
-3.1K
Growth Rate:
-13.69%
Sponsored by
APIMart
inference-optimization / dspark-inkling-small
Model's Last Updated: September 04 2026
Introduction of dspark-inkling-small
Model Details of dspark-inkling-small
DSpark drafter for Inkling-Small-NVFP4
A
DSpark
speculative-decoding drafter
trained for the verifier
thinkingmachines/Inkling-Small-NVFP4
(multimodal MoE, NVFP4). DSpark drafts a whole block in one parallel pass
(DFlash-style backbone) and injects intra-block dependency with a lightweight
sequential Markov head, plus a per-position confidence head.
Model details
| Algorithm | DSpark (greedy proposal) |
| Verifier |
thinkingmachines/Inkling-Small-NVFP4
|
| Backbone layers | 6 |
| Hidden size | 4096 |
| Attention heads / KV heads | 32 / 8 |
| Draft vocab size | 201024 (full padded vocab) |
| Block size | 16 |
| Markov rank | 256 (vanilla) |
| Confidence head | enabled (with Markov) |
| Aux hidden-state layers | 2, 10, 18, 26, 34, 39 |
| dtype | bfloat16 |
Acceptance evaluation
Inference acceptance measured with vLLM spec-decode over 9 task subsets (~300k drafts total). Mean acceptance length (AL) = expected number of tokens accepted per draft step (higher is better; block size 16).
| Subset | Acceptance length | pos-0 | pos-1 | pos-2 | pos-3 | pos-4 | pos-5 | pos-6 | pos-7 |
|---|---|---|---|---|---|---|---|---|---|
| math_reasoning | 4.55 | 0.776 | 0.611 | 0.476 | 0.376 | 0.297 | 0.232 | 0.182 | 0.141 |
| HumanEval | 3.90 | 0.740 | 0.533 | 0.389 | 0.288 | 0.215 | 0.165 | 0.129 | 0.101 |
| rag | 3.73 | 0.736 | 0.537 | 0.394 | 0.288 | 0.212 | 0.156 | 0.113 | 0.083 |
| translation | 3.56 | 0.736 | 0.535 | 0.382 | 0.269 | 0.188 | 0.132 | 0.094 | 0.067 |
| summarization | 3.40 | 0.731 | 0.530 | 0.375 | 0.265 | 0.178 | 0.119 | 0.077 | 0.050 |
| tool_call | 3.20 | 0.707 | 0.484 | 0.328 | 0.222 | 0.148 | 0.100 | 0.067 | 0.045 |
| writing | 3.11 | 0.679 | 0.447 | 0.296 | 0.198 | 0.136 | 0.095 | 0.068 | 0.050 |
| qa | 3.10 | 0.684 | 0.464 | 0.318 | 0.216 | 0.146 | 0.098 | 0.065 | 0.043 |
| question | 3.09 | 0.679 | 0.446 | 0.294 | 0.197 | 0.136 | 0.093 | 0.067 | 0.048 |
| Average | 3.40 | 0.709 | 0.493 | 0.344 | 0.241 | 0.170 | 0.121 | 0.087 | 0.063 |
Draft-weighted average across all subsets shown in the last row.
Per-position acceptance rate, draft-weighted mean over all subsets:
Full per-position numbers are in
acceptance_eval.csv
.
Usage (vLLM)
Serve the verifier with this drafter as the speculative model:
vllm serve thinkingmachines/Inkling-Small-NVFP4 \
--speculative-config '{"model": "<this-repo>", "num_speculative_tokens": 16}' \
--enforce-eager \
--trust-remote-code
Required vLLM patches (both open at time of writing):
- vllm-project/vllm#52184 — adds Eagle3 hidden-state extraction support for Inkling (the verifier must expose the aux hidden states this drafter consumes).
- vllm-project/vllm#55133 — treats a full/padded-vocab draft (draft
vocab_size201024 vs. the verifier's logical 200058) as an identity map, so nod2tmapping is required to load it.Also note: Inkling currently requires
--enforce-eager(CUDA-graph capture of the custom ops crashes).
Training
Trained with
speculators
for 3
epochs (
checkpoint_best
= best validation epoch). Validation at the selected
checkpoint: accept_len 3.67, accept_rate 0.40, pos-0 acc 0.776. Loss
{"ce": 0.1, "tv": 0.9}
, lr 1e-4, seq len 8192, block size 16.
Training curves (run
dspark_inkling_small_v2
, logged with trackio):
Provenance / reproducibility:
-
train_command.txt— exact training command, git SHA, and package versions (speculators 0.7.0.dev155, vllm 0.27.1, transformers 5.15.0, torch 2.13.0+cu130, compressed-tensors 0.17.0). -
training_state.json— epoch / global step of this checkpoint. -
val_metrics.json— full validation metrics.
Runs of inference-optimization dspark-inkling-small on huggingface.co
87
Total runs
0
24-hour runs
2
3-day runs
7
7-day runs
87
30-day runs
More Information About dspark-inkling-small huggingface.co Model
Url of dspark-inkling-small
dspark-inkling-small huggingface.co Url
Provider of dspark-inkling-small huggingface.co
Other API from inference-optimization
Updated:June 16 2026
Total runs:
16.7K
Run Growth:
-1.9K
Growth Rate:
-11.51%
Total runs:
8.2K
Run Growth:
-7.8K
Growth Rate:
-95.80%
Total runs:
7.4K
Run Growth:
-380
Growth Rate:
-5.13%
Total runs:
5.7K
Run Growth:
-14.0K
Growth Rate:
-243.95%
Total runs:
5.4K
Run Growth:
2.1K
Growth Rate:
38.58%
Total runs:
4.5K
Run Growth:
1.5K
Growth Rate:
31.67%
Total runs:
4.5K
Run Growth:
-962
Growth Rate:
-21.30%
Total runs:
2.6K
Run Growth:
1.3K
Growth Rate:
48.53%
Total runs:
1.7K
Run Growth:
1.6K
Growth Rate:
98.32%
Total runs:
1.2K
Run Growth:
-84.8K
Growth Rate:
-6954.63%
Total runs:
1.0K
Run Growth:
-891
Growth Rate:
-85.84%
Total runs:
702
Run Growth:
76
Growth Rate:
10.83%
Total runs:
546
Run Growth:
546
Growth Rate:
100.00%
Total runs:
528
Run Growth:
282
Growth Rate:
53.41%
Total runs:
498
Run Growth:
347
Growth Rate:
85.89%
Total runs:
443
Run Growth:
443
Growth Rate:
100.00%
Total runs:
435
Run Growth:
0
Growth Rate:
0.00%
Total runs:
384
Run Growth:
-316
Growth Rate:
-82.29%
Total runs:
330
Run Growth:
26
Growth Rate:
7.88%
Total runs:
312
Run Growth:
312
Growth Rate:
100.00%
Total runs:
287
Run Growth:
287
Growth Rate:
100.00%
Total runs:
245
Run Growth:
245
Growth Rate:
100.00%
Total runs:
199
Run Growth:
199
Growth Rate:
100.00%
Total runs:
186
Run Growth:
19
Growth Rate:
10.22%
Total runs:
182
Run Growth:
180
Growth Rate:
98.90%
Total runs:
168
Run Growth:
168
Growth Rate:
100.00%
Total runs:
152
Run Growth:
152
Growth Rate:
100.00%
Total runs:
151
Run Growth:
19
Growth Rate:
12.58%
Total runs:
144
Run Growth:
105
Growth Rate:
72.92%
Total runs:
126
Run Growth:
126
Growth Rate:
100.00%
Total runs:
123
Run Growth:
123
Growth Rate:
100.00%
Total runs:
120
Run Growth:
120
Growth Rate:
100.00%
Total runs:
118
Run Growth:
80
Growth Rate:
100.00%
Total runs:
115
Run Growth:
4
Growth Rate:
3.48%
Total runs:
108
Run Growth:
25
Growth Rate:
23.15%
Total runs:
90
Run Growth:
52
Growth Rate:
57.78%
Total runs:
89
Run Growth:
89
Growth Rate:
100.00%
Total runs:
84
Run Growth:
60
Growth Rate:
71.43%
Total runs:
80
Run Growth:
80
Growth Rate:
100.00%
Total runs:
80
Run Growth:
25
Growth Rate:
31.25%
Total runs:
79
Run Growth:
39
Growth Rate:
49.37%
Total runs:
75
Run Growth:
0
Growth Rate:
0.00%
Total runs:
74
Run Growth:
0
Growth Rate:
0.00%
Total runs:
72
Run Growth:
46
Growth Rate:
63.89%
Total runs:
69
Run Growth:
37
Growth Rate:
53.62%
Total runs:
68
Run Growth:
6
Growth Rate:
8.82%
Total runs:
64
Run Growth:
30
Growth Rate:
46.88%
Total runs:
64
Run Growth:
39
Growth Rate:
60.94%
Total runs:
63
Run Growth:
32
Growth Rate:
50.79%
Total runs:
60
Run Growth:
60
Growth Rate:
100.00%
Total runs:
56
Run Growth:
27
Growth Rate:
48.21%
Total runs:
56
Run Growth:
31
Growth Rate:
55.36%
Total runs:
55
Run Growth:
31
Growth Rate:
56.36%
Total runs:
54
Run Growth:
0
Growth Rate:
0.00%
Total runs:
53
Run Growth:
27
Growth Rate:
50.94%
Total runs:
52
Run Growth:
2
Growth Rate:
3.85%
Total runs:
49
Run Growth:
-67
Growth Rate:
-136.73%
Total runs:
41
Run Growth:
-5
Growth Rate:
-13.51%
Total runs:
40
Run Growth:
-15
Growth Rate:
-37.50%
Total runs:
35
Run Growth:
35
Growth Rate:
100.00%
Total runs:
34
Run Growth:
34
Growth Rate:
100.00%
Total runs:
33
Run Growth:
-17
Growth Rate:
-51.52%
Total runs:
32
Run Growth:
6
Growth Rate:
18.75%
Total runs:
30
Run Growth:
11
Growth Rate:
36.67%
Total runs:
30
Run Growth:
30
Growth Rate:
100.00%
Total runs:
29
Run Growth:
15
Growth Rate:
51.72%
Total runs:
29
Run Growth:
25
Growth Rate:
86.21%
Total runs:
28
Run Growth:
28
Growth Rate:
100.00%
Total runs:
27
Run Growth:
-2
Growth Rate:
-7.14%
Total runs:
26
Run Growth:
25
Growth Rate:
100.00%
Total runs:
26
Run Growth:
3
Growth Rate:
11.54%
Total runs:
25
Run Growth:
-36
Growth Rate:
-144.00%
Total runs:
25
Run Growth:
25
Growth Rate:
100.00%
Total runs:
25
Run Growth:
-104
Growth Rate:
-416.00%
Total runs:
24
Run Growth:
-32
Growth Rate:
-133.33%
Total runs:
23
Run Growth:
12
Growth Rate:
52.17%
Total runs:
22
Run Growth:
10
Growth Rate:
45.45%
Total runs:
21
Run Growth:
-5
Growth Rate:
-20.00%
Total runs:
20
Run Growth:
18
Growth Rate:
100.00%
Total runs:
20
Run Growth:
-11
Growth Rate:
-55.00%
Total runs:
20
Run Growth:
20
Growth Rate:
100.00%
Total runs:
18
Run Growth:
5
Growth Rate:
27.78%
Total runs:
18
Run Growth:
-15
Growth Rate:
-83.33%
Total runs:
17
Run Growth:
-57
Growth Rate:
-335.29%
Total runs:
17
Run Growth:
17
Growth Rate:
100.00%
Total runs:
17
Run Growth:
6
Growth Rate:
35.29%