Sponsored by Vmake Labs Video Enhancer
Toolify Toolify logo
AI Models
Qwen3-30B-A3B-speculator.dflash

inference-optimization / Qwen3-30B-A3B-speculator.dflash

Run on huggingface.co
huggingface.co
Total runs: 287
24-hour runs: 0
7-day runs: 82
30-day runs: 287
Model's Last Updated: July 14 2026

Introduction of Qwen3-30B-A3B-speculator.dflash

Model Details of Qwen3-30B-A3B-speculator.dflash

RedHatAI/Qwen3-30B-A3B-speculator.dflash

This is a DFlash speculator model for Qwen/Qwen3-30B-A3B .

Training Details

This model was trained using the Speculators library on a subset of Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered and the train_sft split of HuggingFaceH4/ultrachat_200k .Responses were regenerated by Qwen/Qwen3-235B-A22B and stored at Qwen3_235B_base .

Commands

Using the Speculators library and the helper scripts provided in the repo.

Prepare data 
# In virtual environment with speculators installed
python scripts/prepare_data.py \
  --model Qwen/Qwen3-30B-A3B
  --data ./regenerated_data.jsonl \
  --output ./output \
  --assistant-pattern "<\|im_start\|>assistant\s*([\s\S]*?)<\|im_end\|>" \
  --seq-length 16384
Launch vLLM 
# In (separate) virutal environment with vllm installed
CUDA_VISIBLE_DEVICES=0,1 vllm_venv/bin/python scripts/launch_vllm.py \
  Qwen/Qwen3-30B-A3B \
  --target-layer-ids 1 12 23 34 45 \
  --max-model-len  32768 \
  --max-num-batched-tokens 32768\
  --tensor-parallel-size 2 \
  --no-enable-chunked-prefill
Launch training

Must be run once vLLM has finished launching and is running in the background. 

# In virtual environment with speculators installed
CUDA_VISIBLE_DEVICES=2,3 torchrun \
  --standalone \
  --nproc_per_node 2 \
  scripts/train.py \
  --verifier-name-or-path Qwen/Qwen3-30B-A3B \     
  --data-path ./output \    
  --on-missing generate \    
  --on-generate delete \    
  --scheduler-type cosine \    
  --draft-vocab-size 32000 \    
  --max-anchors 1024 \    
  --target-layer-ids 1 12 23 34 45 \
  --speculator-type dflash \    
  --num-layers 5 \    
  --logger trackio  \    
  --lr 0.0006 \    
  --epochs 5 \    
  --sliding-window 2048 \    
  --sliding-window-indices 0 1 2 3 4 \    
  --draft-hidden-act silu
Model Specifications
Base Model Qwen/Qwen3-30B-A3B
Chat Template Qwen/Qwen3-30B-A3B (use /chat/completions endpoint)
Format Safetensors
License Apache 2.0
Validation Hardware Nvidia A100
Deployment 
# Install vLLM from the required PR
pip install git+https://github.com/vllm-project/vllm.git     
                                                                                                                                                                                                                                                                                                          
# Deploy with speculative decoding                                                                                                                                                                                                                                                                        
vllm serve Qwen/Qwen3-30B-A3B \                                                                                                                                                                                                                                                                                
    --tensor-parallel-size 2 \                                                                                                                                                                                                                                                                            
    --max-num-batched-tokens 32768 \
    --attention-backend FLASH_ATTN \ 
    --speculative-config '{                                                                                                                                                                                                                                                                               
        "model": "RedHatAI/Qwen3-30B-A3B-speculator.dflash",                                                                                                                                                                                                                                                   
        "num_speculative_tokens": 15,                                                                                                                                                                                                                                                                      
        "method": "dflash"                                                                                                                                                                                                                                                                                
    }'
Preliminary Evaluations

Per-position token acceptance rates across datasets:
(with reasoning enabled)

Dataset Pos 1 Pos 2 Pos 3 Pos 4 Pos 5 Pos 6 Pos 7 Pos 8 Pos 9 Pos 10 Pos 11 Pos 12 Pos 13 Pos 14 Pos 15 Avg Acceptance Rate
HumanEval 75.89% 50.00% 28.33% 17.07% 10.07% 5.91% 3.59% 2.35% 1.48% 1.23% 1.10% 1.05% 1.04% 1.02% 1.01% 13.40%
math_reasoning 81.55% 58.93% 41.26% 27.69% 18.14% 11.01% 6.08% 3.07% 1.41% 0.58% 0.18% 0.06% 0.01% 0.00% 0.00% 16.64%
qa 67.88% 39.38% 20.96% 10.35% 4.87% 2.20% 0.87% 0.38% 0.11% 0.04% 0.01% 0.00% 0.00% 0.00% 0.00% 9.81%
question 73.53% 46.00% 26.97% 15.37% 8.62% 4.62% 2.30% 1.05% 0.41% 0.13% 0.03% 0.00% 0.00% 0.00% 0.00% 11.93%
rag 72.66% 45.06% 25.47% 13.48% 6.83% 3.32% 1.39% 0.59% 0.19% 0.04% 0.01% 0.00% 0.00% 0.00% 0.00% 11.30%
summarization 66.98% 36.52% 17.98% 8.25% 3.51% 1.37% 0.49% 0.12% 0.02% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 9.03%
tool_call 71.85% 44.02% 24.62% 13.09% 6.65% 3.49% 1.74% 0.73% 0.29% 0.14% 0.01% 0.00% 0.00% 0.00% 0.00% 11.12%
translation 67.64% 40.76% 21.68% 9.99% 4.66% 2.11% 0.90% 0.39% 0.11% 0.02% 0.00% 0.00% 0.00% 0.00% 0.00% 9.86%
writing 73.99% 46.44% 27.41% 15.78% 8.90% 4.72% 2.36% 1.13% 0.45% 0.19% 0.07% 0.01% 0.00% 0.00% 0.00% 12.09%
References

Paper : DFlash: Block Diffusion for Flash Speculative Decoding

Runs of inference-optimization Qwen3-30B-A3B-speculator.dflash on huggingface.co

287
Total runs
0
24-hour runs
33
3-day runs
82
7-day runs
287
30-day runs

More Information About Qwen3-30B-A3B-speculator.dflash huggingface.co Model

More Qwen3-30B-A3B-speculator.dflash license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen3-30B-A3B-speculator.dflash huggingface.co

Qwen3-30B-A3B-speculator.dflash huggingface.co is an AI model on huggingface.co that provides Qwen3-30B-A3B-speculator.dflash's model effect (), which can be used instantly with this inference-optimization Qwen3-30B-A3B-speculator.dflash model. huggingface.co supports a free trial of the Qwen3-30B-A3B-speculator.dflash model, and also provides paid use of the Qwen3-30B-A3B-speculator.dflash. Support call Qwen3-30B-A3B-speculator.dflash model through api, including Node.js, Python, http.

Qwen3-30B-A3B-speculator.dflash huggingface.co Url

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-speculator.dflash

inference-optimization Qwen3-30B-A3B-speculator.dflash online free

Qwen3-30B-A3B-speculator.dflash huggingface.co is an online trial and call api platform, which integrates Qwen3-30B-A3B-speculator.dflash's modeling effects, including api services, and provides a free online trial of Qwen3-30B-A3B-speculator.dflash, you can try Qwen3-30B-A3B-speculator.dflash online for free by clicking the link below.

inference-optimization Qwen3-30B-A3B-speculator.dflash online free url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-speculator.dflash

Qwen3-30B-A3B-speculator.dflash install

Qwen3-30B-A3B-speculator.dflash is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-30B-A3B-speculator.dflash on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-30B-A3B-speculator.dflash install, users can directly use Qwen3-30B-A3B-speculator.dflash installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen3-30B-A3B-speculator.dflash install url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-speculator.dflash

Url of Qwen3-30B-A3B-speculator.dflash

Provider of Qwen3-30B-A3B-speculator.dflash huggingface.co

inference-optimization
ORGANIZATIONS
huggingface.co

Other API from inference-optimization