inference-optimization / Qwen3-8B-DFlash-PerfectBlend-FP8-W8A8

Blend8-FP8-W8A8 — Static FP8 W8A8 with PerfectBlend calibration

Blend8 is a quantized DFlash drafter for the Qwen3-8B target , derived from RedHatAI/Qwen3-8B-speculator.dflash . This repository contains the drafter component; it is not a standalone chat model.

Variant

• Quantization: FP8 static W8A8.
• Calibration: Real PerfectBlend calibration using aligned Qwen3-8B hidden states, token inputs, and loss masks. The cache had 2,027 prepared examples, sequence length 2,048, and calibration seed 0. The cache and prompts are not included.
• Calibration seed: 0.
• Quantization settings and source revisions: quant_run_manifest.json .

Use with vLLM

Pair this drafter with the Qwen3-8B target and a DFlash-capable vLLM build:

vllm serve Qwen/Qwen3-8B \
  --spec-model inference-optimization/Qwen3-8B-DFlash-Blend8-FP8-W8A8 \
  --spec-tokens 7 \
  --spec-method dflash

config.py provides the custom drafter configuration. The experiment's serving command and runtime patch are in provenance/evaluation/ .

Reproducibility

The manifests are included at the repository root. provenance/ contains the source drafter's captured train_command.txt , a quantization command explicitly marked as reconstructed, the quantizer and calibration source snapshot, the vLLM command and patch, both target and drafter checkpoint hashes, and the nine per-subset evaluation commands. The selected seed-0 checkpoint is the same checkpoint used in the 2026-09-24 all-subset evaluation.

The PerfectBlend preparation, prompts, and hidden-state cache remain local because the prepared prompts are not redistributable. The cache sample counts and content hashes are recorded in calibration_manifest.json ; no prompts or hidden-state tensors are uploaded.

The source drafter lists Apache-2.0 licensing on its Hugging Face model card .