Sponsored by AdsCreator.com
Toolify Toolify logo
AI Models
Qwen3-30B-A3B-Instruct-2507-speculator.dflash

inference-optimization / Qwen3-30B-A3B-Instruct-2507-speculator.dflash

Run on huggingface.co
huggingface.co
Total runs: 546
24-hour runs: 0
7-day runs: 300
30-day runs: 546
Model's Last Updated: July 09 2026

Introduction of Qwen3-30B-A3B-Instruct-2507-speculator.dflash

Model Details of Qwen3-30B-A3B-Instruct-2507-speculator.dflash

RedHatAI/Qwen3-30B-A3B-Instruct-2507-speculator.dflash

This is a DFlash speculator model for Qwen/Qwen3-30B-A3B-Instruct-2507 .

Training Details

This model was trained using the Speculators library on a subset of Magpie-Align/Magpie-Llama-3.1-Pro-300K-Filtered and the train_sft split of HuggingFaceH4/ultrachat_200k . Responses were regenerated by Qwen3-30B-A3B-Instruct-2507 (with reasoning). Training compute for this model was sponsored by Modal .

Commands

Using the Speculators library and the helper scripts provided in the repo.

Prepare data 
# In virtual environment with speculators installed
python scripts/prepare_data.py \
  --model Qwen/Qwen3-30B-A3B-Instruct-2507
  --data ./regenerated_data.jsonl \
  --output ./output \
  --seq-length 16384
Launch vLLM 
# In (separate) virutal environment with vllm installed
CUDA_VISIBLE_DEVICES=0,1 vllm_venv/bin/python scripts/launch_vllm.py \
  Qwen/Qwen3-30B-A3B-Instruct-2507 \
  --target-layer-ids 1 12 23 34 45 \
  --max-model-len  32768 \
  --max-num-batched-tokens 32768\
  --tensor-parallel-size 2 \
  --no-enable-chunked-prefill
Launch training

Must be run once vLLM has finished launching and is running in the background. 

# In virtual environment with speculators installed
CUDA_VISIBLE_DEVICES=2,3 torchrun \
  --standalone \
  --nproc_per_node 2 \
  scripts/train.py \
  --verifier-name-or-path Qwen/Qwen3-30B-A3B-Instruct-2507 \     
  --data-path ./output \    
  --on-missing generate \    
  --on-generate delete \    
  --scheduler-type cosine \    
  --draft-vocab-size 32000 \    
  --max-anchors 1024 \    
  --target-layer-ids 1 12 23 34 45 \
  --speculator-type dflash \    
  --num-layers 5 \    
  --logger trackio  \    
  --lr 0.0006 \    
  --epochs 5 \    
  --sliding-window 2048 \    
  --sliding-window-indices 0 1 2 3 4 \    
  --draft-hidden-act silu
Model Specifications
Base Model Qwen/Qwen3-30B-A3B-Instruct-2507
Chat Template Qwen/Qwen3-30B-A3B-Instruct-2507 (use /chat/completions endpoint)
Format Safetensors
License Apache 2.0
Validation Hardware Nvidia A100
Deployment 
# Install vLLM from the required PR
pip install git+https://github.com/vllm-project/vllm.git     
                                                                                                                                                                                                                                                                                                          
# Deploy with speculative decoding                                                                                                                                                                                                                                                                        
vllm serve Qwen/Qwen3-30B-A3B-Instruct-2507 \                                                                                                                                                                                                                                                                                
    --tensor-parallel-size 2 \                                                                                                                                                                                                                                                                            
    --max-num-batched-tokens 32768 \
    --attention-backend FLASH_ATTN \ 
    --speculative-config '{                                                                                                                                                                                                                                                                               
        "model": "RedHatAI/Qwen3-30B-A3B-Instruct-2507-speculator.dflash",                                                                                                                                                                                                                                                   
        "num_speculative_tokens": 15,                                                                                                                                                                                                                                                                      
        "method": "dflash"                                                                                                                                                                                                                                                                                
    }'
Preliminary Evaluations

Per-position token acceptance rates across datasets:
(with reasoning enabled)

Dataset Pos 1 Pos 2 Pos 3 Pos 4 Pos 5 Pos 6 Pos 7 Pos 8 Pos 9 Pos 10 Pos 11 Pos 12 Pos 13 Pos 14 Pos 15 Avg Acceptance Rate
HumanEval 80.78% 61.07% 45.17% 33.18% 24.47% 17.83% 12.49% 7.89% 4.39% 1.99% 0.79% 0.24% 0.07% 0.00% 0.00% 19.34%
math_reasoning 84.71% 67.42% 52.57% 40.94% 31.17% 23.41% 17.22% 11.15% 6.45% 2.95% 1.14% 0.46% 0.22% 0.07% 0.00% 22.66%
qa 59.36% 30.69% 15.48% 7.62% 3.46% 1.79% 0.69% 0.30% 0.10% 0.05% 0.00% 0.00% 0.00% 0.00% 0.00% 7.99%
question 66.95% 39.10% 22.94% 14.08% 8.91% 6.07% 3.84% 2.22% 1.23% 0.49% 0.18% 0.05% 0.02% 0.02% 0.00% 11.06%
rag 63.58% 35.87% 18.79% 9.92% 4.99% 2.28% 1.07% 0.38% 0.12% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 9.11%
summarization 60.85% 30.37% 14.09% 6.58% 2.84% 1.32% 0.51% 0.12% 0.01% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 7.79%
tool_call 65.47% 39.84% 24.14% 14.97% 9.09% 5.54% 3.36% 1.87% 0.94% 0.39% 0.12% 0.06% 0.00% 0.00% 0.00% 11.03%
translation 68.00% 34.60% 14.40% 5.40% 1.80% 0.70% 0.20% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 8.30%
writing 65.98% 38.84% 22.62% 14.32% 9.14% 5.90% 3.93% 2.45% 1.22% 0.61% 0.22% 0.07% 0.00% 0.00% 0.00% 11.00%
References

Paper : DFlash: Block Diffusion for Flash Speculative Decoding

Runs of inference-optimization Qwen3-30B-A3B-Instruct-2507-speculator.dflash on huggingface.co

546
Total runs
0
24-hour runs
154
3-day runs
300
7-day runs
546
30-day runs

More Information About Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co Model

More Qwen3-30B-A3B-Instruct-2507-speculator.dflash license Visit here:

https://choosealicense.com/licenses/apache-2.0

Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co

Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co is an AI model on huggingface.co that provides Qwen3-30B-A3B-Instruct-2507-speculator.dflash's model effect (), which can be used instantly with this inference-optimization Qwen3-30B-A3B-Instruct-2507-speculator.dflash model. huggingface.co supports a free trial of the Qwen3-30B-A3B-Instruct-2507-speculator.dflash model, and also provides paid use of the Qwen3-30B-A3B-Instruct-2507-speculator.dflash. Support call Qwen3-30B-A3B-Instruct-2507-speculator.dflash model through api, including Node.js, Python, http.

Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co Url

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-Instruct-2507-speculator.dflash

inference-optimization Qwen3-30B-A3B-Instruct-2507-speculator.dflash online free

Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co is an online trial and call api platform, which integrates Qwen3-30B-A3B-Instruct-2507-speculator.dflash's modeling effects, including api services, and provides a free online trial of Qwen3-30B-A3B-Instruct-2507-speculator.dflash, you can try Qwen3-30B-A3B-Instruct-2507-speculator.dflash online for free by clicking the link below.

inference-optimization Qwen3-30B-A3B-Instruct-2507-speculator.dflash online free url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-Instruct-2507-speculator.dflash

Qwen3-30B-A3B-Instruct-2507-speculator.dflash install

Qwen3-30B-A3B-Instruct-2507-speculator.dflash is an open source model from GitHub that offers a free installation service, and any user can find Qwen3-30B-A3B-Instruct-2507-speculator.dflash on GitHub to install. At the same time, huggingface.co provides the effect of Qwen3-30B-A3B-Instruct-2507-speculator.dflash install, users can directly use Qwen3-30B-A3B-Instruct-2507-speculator.dflash installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

Qwen3-30B-A3B-Instruct-2507-speculator.dflash install url in huggingface.co:

https://huggingface.co/inference-optimization/Qwen3-30B-A3B-Instruct-2507-speculator.dflash

Url of Qwen3-30B-A3B-Instruct-2507-speculator.dflash

Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co Url

Provider of Qwen3-30B-A3B-Instruct-2507-speculator.dflash huggingface.co

inference-optimization
ORGANIZATIONS
huggingface.co

Other API from inference-optimization