lued / Qwen3.8-27B-DFlash2-W8

This is a numerical W8A16 quantization of the z-lab/Qwen3.8-27B-DFlash2 drafter (mirror of incoai/Qwen3.8-27B-DFlash2 ). Model credit belongs to the z-lab / incoai DFlash2 team and the Qwen base; this repository changes numerics only.

This is a draft model, not a standalone LLM . It is the speculative-drafting component of DFlash2 decoding: it proposes tokens that the target checkpoint verifies before any are emitted. It never emits unverified tokens. Serve it only as the --speculative-config drafter of a DFlash2-compatible target.

What this is

The DFlash2 drafter ( PR vllm#52816 , open as of 2026-08-19) is a 5-layer, block-diffusion draft model with a top-K candidate selector, trained to predict the target's next tokens at fixed layers. Quantizing it W8A16 drops it from 3.58 GiB (BF16) to 2.02 GiB with no measurable effect on drafting (W8 vs BF16 drafter acceptance is equal: 2.65 vs 2.72/draft at the llama-benchy prose floor, same session).

The quantization was produced by the int8-w8a16-dflash2 family toolkit ( tools/quantize_draft_w8a16.py ); the exact target set and ignore list are recorded in this repository's recipe.yaml and config.json .

Usage

Serve it as the DFlash2 drafter of the matching quantized target lued/Qwen3.8-27B-INT8-W8A16-DFlash2 by repo ID (both resolve from your HF cache):

{"method": "dflash", "model": "lued/Qwen3.8-27B-DFlash2-W8", "num_speculative_tokens": 7}

• Spec window: 7 tokens (block 8), the drafter's trained design; shorter windows measure slower.
• vLLM: requires the vendored PR 52816 patch set from the club-3090 repo . See the target card's "Serving" section for the launch.
• Acceptance is workload-shaped, not a fixed number: ~6.0 tokens/draft on math, ~3.3 on long prose (mean acceptance length including the bonus token, same definition as the z-lab card). See the target card's "Speculative acceptance" section for the envelope and the z-lab comparison.