Sponsored by Vmake Labs Video Enhancer
Toolify Toolify logo
AI Models
Qwen3-8B-DFlash-FP8-BLOCK

inference-optimization / Qwen3-8B-DFlash-FP8-BLOCK

Run on huggingface.co
huggingface.co
Total runs: 59
24-hour runs: 3
7-day runs: 15
30-day runs: 59
Model's Last Updated: September 28 2026
text-generation

Introduction of Qwen3-8B-DFlash-FP8-BLOCK

Model Details of Qwen3-8B-DFlash-FP8-BLOCK

Block8-FP8-BLOCK — Data-free FP8 Block quantization

Block8 is a quantized DFlash drafter for the Qwen3-8B target , derived from RedHatAI/Qwen3-8B-speculator.dflash at pinned revision 1a11b170eb65c8a62c80ecd01dfe22a5907298e6 . This repository contains the drafter component; it is not a standalone chat model.

Variant
  • Quantization: FP8 Block ( model_free_ptq(scheme="FP8_BLOCK") with static 128x128 blocks and dynamic 128 groups).
  • Calibration: Data-free quantization; no calibration data or samples were used.
  • Source snapshot: Pinned newer BF16 drafter snapshot 1a11b170eb65c8a62c80ecd01dfe22a5907298e6 (block size 16, draft vocabulary 151,936, sliding-window attention).
Use with vLLM

Pair this drafter with the Qwen3-8B target and a DFlash-capable vLLM build: 

vllm serve Qwen/Qwen3-8B \
  --spec-model inference-optimization/Qwen3-8B-DFlash-FP8-BLOCK \
  --spec-tokens 7 \
  --spec-method dflash

config.py provides the custom drafter configuration. Serving command and runtime patch are preserved in provenance/evaluation/ .

Evaluation Performance

Evaluated with 7 speculative tokens per draft event against the Qwen/Qwen3-8B target:

Benchmark Mean Acceptance Length ($L = 1 + A/D$) Mean Accepted Ratio ($A/P$)
SPEED-Bench Qualitative (11 categories, 880 requests) 3.1535 tokens 30.76%
RedHatAI / speculator_benchmarks (9 subsets, 924 requests) 3.1410 tokens 30.59%
Reproducibility & Provenance

Full provenance and reproducibility artifacts are preserved:

  • Root directory contains model weights, configuration, tokenizer, quantization manifests, and recipe.
  • provenance/ contains:
    • train_command.txt : Source model training command.
    • train_command_scope.txt : Clarification of source drafter snapshot lineage.
    • quantization_command.txt : Exact command and environment used to run quantization.
    • quantization_source/ : Source code snapshot of quantizer utilities.
    • evaluation/ : Contains vLLM serving commands ( vllm_command.txt ), vLLM runtime patch ( vllm.patch ), speculators patch ( speculators.patch ), target/drafter SHA-256 hashes, and per-subset evaluation commands.
    • publication_sha256.txt : SHA-256 hashes of all published files.

The source drafter lists Apache-2.0 licensing on its Hugging Face model card .

Runs of inference-optimization Qwen3-8B-DFlash-FP8-BLOCK on huggingface.co

59
Total runs
3
24-hour runs
8
3-day runs
15
7-day runs
59
30-day runs

More Information About Qwen3-8B-DFlash-FP8-BLOCK huggingface.co Model

More Qwen3-8B-DFlash-FP8-BLOCK license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen3-8B-DFlash-FP8-BLOCK huggingface.co

Qwen3-8B-DFlash-FP8-BLOCK huggingface.co is an AI model on huggingface.co that provides Qwen3-8B-DFlash-FP8-BLOCK's model effect (), which can be used instantly with this inference-optimization Qwen3-8B-DFlash-FP8-BLOCK model. huggingface.co supports a free trial of the Qwen3-8B-DFlash-FP8-BLOCK model, and also provides paid use of the Qwen3-8B-DFlash-FP8-BLOCK. Support call Qwen3-8B-DFlash-FP8-BLOCK model through api, including Node.js, Python, http.

Qwen3-8B-DFlash-FP8-BLOCK huggingface.co Url

https://huggingface.co/inference-optimization/Qwen3-8B-DFlash-FP8-BLOCK

inference-optimization Qwen3-8B-DFlash-FP8-BLOCK online free

Qwen3-8B-DFlash-FP8-BLOCK huggingface.co is an online trial and call api platform, which integrates Qwen3-8B-DFlash-FP8-BLOCK's modeling effects, including api services, and provides a free online trial of Qwen3-8B-DFlash-FP8-BLOCK, you can try Qwen3-8B-DFlash-FP8-BLOCK online for free by clicking the link below.

inference-optimization Qwen3-8B-DFlash-FP8-BLOCK online free url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-8B-DFlash-FP8-BLOCK

Qwen3-8B-DFlash-FP8-BLOCK install

Qwen3-8B-DFlash-FP8-BLOCK is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-8B-DFlash-FP8-BLOCK on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-8B-DFlash-FP8-BLOCK install, users can directly use Qwen3-8B-DFlash-FP8-BLOCK installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen3-8B-DFlash-FP8-BLOCK install url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-8B-DFlash-FP8-BLOCK

Url of Qwen3-8B-DFlash-FP8-BLOCK

Provider of Qwen3-8B-DFlash-FP8-BLOCK huggingface.co

inference-optimization
ORGANIZATIONS
huggingface.co

Other API from inference-optimization