Inferact / Kimi-K3-DSpark-Block5

Kimi-K3 DSpark — Block 5

A DSpark draft model for speculative decoding with moonshotai/Kimi-K3 . Three full-causal MLA layers, conditioned on hidden states extracted from three of the target's layers and trained at block_size=5 : one forward pass drafts the full five-token block.

Training

The draft completed a four-epoch run with speculators , the speculative-decoding library from the vLLM project , against Kimi-K3 hidden states streamed live from vLLM .

Acceptance

Evaluated against Kimi-K3 on 8×B300 (TP8), drafting 5 tokens with probabilistic sampling and block rejection at temperature 1.0 and top-p 0.95. All 1,604 requests succeeded. The aggregate is computed from summed counters, not an average of arm rates.

The separate five-shot GSM8K accuracy check scored 96.51% exact match on all 1,319 examples with zero request errors.

The full 1M context window is supported through YaRN , evaluated on every row of the BEAM 100K, 500K, and 1M splits above. The fixed 1M acceptance corpus left-truncates oldest turns in 14 of 35 over-limit source conversations to keep prompt plus output within 1,048,576 tokens; its longest measured prompt is 1,039,871 tokens. Two successful completions were flagged as degenerate by the client (BEAM 500K conversation 4 and BEAM 1M conversation 18). Full counters, per-position rates, and quality flags are in benchmark_results.json .

Serving

vllm serve moonshotai/Kimi-K3 \
  --trust-remote-code \
  --tensor-parallel-size 8 \
  --max-model-len 1048576 \
  --speculative-config '{
    "method": "dspark",
    "model": "Inferact/Kimi-K3-DSpark-Block5",
    "num_speculative_tokens": 5,
    "attention_backend": "FLASHINFER_MLA",
    "draft_sample_method": "probabilistic",
    "rejection_sample_method": "block"
  }'