stepfun-ai / Step-3.5-Flash

Step 3.5 Flash

1. Introduction

Step 3.5 Flash is our most capable open-source foundation model, engineered to deliver frontier reasoning and agentic capabilities with exceptional efficiency. Built on a sparse Mixture of Experts (MoE) architecture, it selectively activates only 11B of its 196B parameters per token. This "intelligence density" allows it to rival the reasoning depth of top-tier proprietary models, while maintaining the agility required for real-time interaction.

2. Key Capabilities

• Deep Reasoning at Speed : While chatbots are built for reading, agents must reason fast. Powered by 3-way Multi-Token Prediction (MTP-3), Step 3.5 Flash achieves a generation throughput of 100–300 tok/s in typical usage (peaking at 350 tok/s for single-stream coding tasks). This allows for complex, multi-step reasoning chains with immediate responsiveness.

• A Robust Engine for Coding & Agents : Step 3.5 Flash is purpose-built for agentic tasks, integrating a scalable RL framework that drives consistent self-improvement. It achieves 74.4% on SWE-bench Verified and 51.0% on Terminal-Bench 2.0 , proving its ability to handle sophisticated, long-horizon tasks with unwavering stability.

• Efficient Long Context : The model supports a cost-efficient 256K context window by employing a 3:1 Sliding Window Attention (SWA) ratio—integrating three SWA layers for every full-attention layer. This hybrid approach ensures consistent performance across massive datasets or long codebases while significantly reducing the computational overhead typical of standard long-context models.

• Accessible Local Deployment : Optimized for accessibility, Step 3.5 Flash brings elite-level intelligence to local environments. It runs securely on high-end consumer hardware (e.g., Mac Studio M4 Max, NVIDIA DGX Spark), ensuring data privacy without sacrificing performance.

3. Performance

Step 3.5 Flash delivers performance parity with leading closed-source systems while remaining open and efficient.

Performance of Step 3.5 Flash measured across Reasoning , Coding , and Agency . Open-source models (left) are sorted by their total parameter count, while top-tier proprietary models are shown on the right. xbench-DeepSearch scores are sourced from official publications for consistency. The shadowed bars represent the enhanced performance of Step 3.5 Flash using Parallel Thinking .

Detailed Benchmarks

Notes :

1. "—" indicates the score is not publicly available or not tested.
2. "*" indicates the original score was inaccessible or lower than our reproduced, so we report the evaluation under the same test conditions as Step 3.5 Flash to ensure fair comparability.
3. BrowseComp (with Context Manager) : When the effective context length exceeds a predefined threshold, the agent resets the context and restarts the agent loop. By contrast, Kimi K2.5 and DeepSeek-V3.2 used a "discard-all" strategy.
4. Decoding Cost : Estimates are based on a methodology similar to, but more accurate than, the approach described arxiv.org/abs/2507.19427

4. Architecture Details

Step 3.5 Flash is built on a Sparse Mixture-of-Experts (MoE) transformer architecture, optimized for high throughput and low VRAM usage during inference.

4.1 Technical Specifications

4.2 Mixture of Experts (MoE) Routing

Unlike traditional dense models, Step 3.5 Flash uses a fine-grained routing strategy to maximize efficiency:

• Fine-Grained Experts : 288 routed experts per layer + 1 shared expert (always active).
• Sparse Activation : Only the Top-8 experts are selected per token.
• Result : The model retains the "memory" of a 196B parameter model but executes with the speed of an 11B model.

4.3 Multi-Token Prediction (MTP)

To improve inference speed, we utilize a specialized MTP Head consisting of a sliding-window attention mechanism and a dense Feed-Forward Network (FFN). This module predicts 4 tokens simultaneously in a single forward pass, significantly accelerating inference without degrading quality.

5. Quick Start

You can get started with Step 3.5 Flash in minutes using Cloud API via our supported providers.

5.1 Get Your API Key

Choose a provider and obtain your credentials. OpenRouter now offers free trial for Step 3.5 Flash.

5.2 Setup

Install the standard OpenAI SDK (compatible with both platforms).

pip install --upgrade "openai>=1.0"

Note: OpenRouter supports multiple SDKs. Learn more here .

5.3 Implementation Example

This example shows starting a chat with Step 3.5 Flash.

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_API_KEY",
    base_url="https://api.stepfun.ai/v1", # or "https://openrouter.ai/api/v1"
    # Optional: OpenRouter headers for app rankings
    default_headers={
        "HTTP-Referer": "<YOUR_SITE_URL>", 
        "X-Title": "<YOUR_SITE_NAME>",
    }
)

completion = client.chat.completions.create(
    model="step-3.5-flash", # Use "stepfun/step-3.5-flash" for OpenRouter
    messages=[
        {
            "role": "system",
            "content": "You are an AI chat assistant provided by StepFun. You are good at Chinese, English, and many other languages.",
        },
        {
            "role": "user",
            "content": "Introduce StepFun's artificial intelligence capabilities."
        },
    ],
)

print(completion.choices[0].message.content)

6. Local Deployment

Step 3.5 Flash is optimized for local inference and supports industry-standard backends including vLLM, SGLang, Hugging Face Transformers and llama.cpp.

6.1 vLLM

We recommend using the latest nightly build of vLLM.

1. Install vLLM.

# via Docker
docker pull vllm/vllm-openai:nightly

# or via pip (nightly wheels)
pip install -U vllm --pre \
  --index-url https://pypi.org/simple \
  --extra-index-url https://wheels.vllm.ai/nightly

1. Launch the server.

Note : Full MTP3 support is not yet available in vLLM. We are actively working on a Pull Request to integrate this feature, which is expected to significantly enhance decoding performance.

• For fp8 model

vllm serve <MODEL_PATH_OR_HF_ID> \
  --served-model-name step3p5-flash \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --disable-cascade-attn \
  --reasoning-parser step3p5 \
  --enable-auto-tool-choice \
  --tool-call-parser step3p5 \
  --hf-overrides '{"num_nextn_predict_layers": 1}' \
  --speculative_config '{"method": "step3p5_mtp", "num_speculative_tokens": 1}' \
  --trust-remote-code \
  --quantization fp8

• For bf16 model

vllm serve <MODEL_PATH_OR_HF_ID> \
  --served-model-name step3p5-flash \
  --tensor-parallel-size 8 \
  --enable-expert-parallel \
  --disable-cascade-attn \
  --reasoning-parser step3p5 \
  --enable-auto-tool-choice \
  --tool-call-parser step3p5 \
  --hf-overrides '{"num_nextn_predict_layers": 1}' \
  --speculative_config '{"method": "step3p5_mtp", "num_speculative_tokens": 1}' \
  --trust-remote-code    

6.2 SGLang

1. Install SGLang.

# via Docker
docker pull lmsysorg/sglang:latest
# or from source (pip)
pip install "sglang[all] @ git+https://github.com/sgl-project/sglang.git"

1. Launch the server.

  - For bf16 model
SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
  --model-path <MODEL_PATH_OR_HF_ID> \
  --served-model-name step3p5-flash \
  --tp-size 8 \
  --tool-call-parser step3p5 \
  --reasoning-parser step3p5 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --enable-multi-layer-eagle \
  --host 0.0.0.0 \
  --port 8000

• For fp8 model

SGLANG_ENABLE_SPEC_V2=1
python3 -m sglang.launch_server \
  --model-path <MODEL_PATH_OR_HF_ID> \
  --served-model-name step3p5-flash \
  --tp-size 8 \
  --ep-size 8 \
  --tool-call-parser step3p5 \
  --reasoning-parser step3p5 \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --enable-multi-layer-eagle \
  --host 0.0.0.0 \
  --port 8000

6.3 Transformers (Debug / Verification)

Use this snippet for quick functional verification. For high-throughput serving, use vLLM or SGLang.

from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "<MODEL_PATH_OR_HF_ID>"

# 1. Setup
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
model = AutoModelForCausalLM.from_pretrained(
    MODEL_PATH,
    trust_remote_code=True,
    torch_dtype="auto",
    device_map="auto",
)

# 2. Prepare Input
messages = [{"role": "user", "content": "Explain the significance of the number 42."}]
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

# 3. Generate
generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
output_text = tokenizer.decode(generated_ids[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)

print(output_text)

6.4 llama.cpp

System Requirements

• GGUF Model Weights(int4): 111.5 GB
• Runtime Overhead: ~7 GB
• Minimum VRAM: 120 GB (e.g., Mac studio, DGX-Spark, AMD Ryzen AI Max+ 395)
• Recommended: 128GB unified memory

Steps

1. Use llama.cpp:

git clone [email protected]:stepfun-ai/Step-3.5-Flash.git
cd Step-3.5-Flash/llama.cpp

1. Build llama.cpp on Mac:

cmake -S . -B build-macos \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_METAL=ON \
  -DGGML_ACCELERATE=ON \
  -DLLAMA_BUILD_EXAMPLES=ON \
  -DLLAMA_BUILD_COMMON=ON \
  -DGGML_LTO=ON
cmake --build build-macos -j8

1. Build llama.cpp on DGX-Spark:

cmake -S . -B build-cuda \
  -DCMAKE_BUILD_TYPE=Release \
  -DGGML_CUDA=ON \
  -DGGML_CUDA_GRAPHS=ON \
  -DLLAMA_CURL=OFF \
  -DLLAMA_BUILD_EXAMPLES=ON \
  -DLLAMA_BUILD_COMMON=ON
cmake --build build-cuda -j8

1. Build llama.cpp on AMD Windows:

cmake -S . -B build-vulkan \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLAMA_CURL=OFF \
  -DGGML_OPENMP=ON \
  -DGGML_VULKAN=ON
cmake --build build-vulkan -j8

1. Run with llama-cli

./llama-cli -m step3.5_flash_Q4_K_S.gguf -c 16384 -b 2048 -ub 2048 -fa on --temp 1.0 -p "What's your name?"

1. Test performance with llama-batched-bench:

./llama-batched-bench -m step3.5_flash_Q4_K_S.gguf -c 32768 -b 2048 -ub 2048 -npp 0,2048,8192,16384,32768 -ntg 128 -npl 1

7. Using Step 3.5 Flash on Agent Platforms

7.1 Claude Code & Codex

It's straightforward to add Step 3.5 Flash to the list of models in most coding environments. See below for the instructions for configuring Claude Code and Codex to use Step 3.5 Flash.

7.1.1 Prerequisites

Sign up at StepFun.ai or OpenRouter and grab an API key, as mentioned in the Quick Start.

7.1.2 Environment setup

Claude Code and Codex rely on Node.js. We recommend installing Node.js version > v20. You can install Node via nvm.

Mac/Linux :

# Install nvm on Mac/Linux via curl：
# Step 1
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash

# Copy the full command
export NVM_DIR="$HOME/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"  # This loads nvm
[ -s "$NVM_DIR/bash_completion" ] && \. "$NVM_DIR/bash_completion"

# Users in China can set up npm mirror
config set registry https://registry.npmmirror.com

# Step 2
nvm install v22

# Make sure Node.js is installed
node --version

npm --version

Windows : You can download the installation file ( nvm-setup.exe ) from https://github.com/coreybutler/nvm-windows/releases . Follow the instructions to install nvm. Run nvm commands to make sure it is installed.

7.1.3 Use Step 3.5 Flash on Claude Code

1. Install Claude Code.

# install claude code via npm
npm install -g @anthropic-ai/claude-code

# test if the installation is successful
claude --version 

1. Configure Claude Code.

We support the OpenAI and Anthropic API style for integration into Claude Code.

Note: OpenAI API style here refers to the chat/completions/ format.

We recommend using claude-code-router . For details, see https://github.com/musistudio/claude-code-router .

After Claude Code is installed, install claude-code-router :

# install ccr via npm
npm install -g @musistudio/claude-code-router

# validate it is installed
ccr -v

Add the following configurations to ~/.claude-code-router/config.json .

{
  "PORT": 3456,
  "Providers": [
    {
      "name": "stepfun-api",
      "api_base_url": "https://api.stepfun.com/v1/chat/completions",
      "api_key": "StepFun_API_KEY",
      "models": ["step-3.5-flash"],
      "transformer":{
           "step-3.5-flash": { "use": ["OpenAI"]}
      }
    }
  ],
  "Router": {
    "default": "stepfun-api,step-3.5-flash",
    "background": "stepfun-api,step-3.5-flash",
    "think": "stepfun-api,step-3.5-flash",
    "longContext": "stepfun-api,step-3.5-flash",
    "webSearch": "stepfun-api,step-3.5-flash"
  }
}

You can now start Claude Code:

# Start Claude
ccr code 

# restart ccr if configs are changed
ccr restart 

7.1.4 Use Step 3.5 Flash on Codex

1. Install Codex

# Install codex via npm
npm install -g @openai/codex

# Test if it is installed
codex --version

1. Configure Codex Add the following settings to ~/.codex/config.toml , keeping the rest of the settings as they are.

model="step-3.5-flash"
model_provider = "stepfun-chat"
preferred_auth_method = "apikey"

# configure the provider
[model_providers.stepfun-chat]
name = "OpenAI using response"
base_url = "https://api.stepfun.com/v1"
env_key = "OPENAI_API_KEY"
wire_api = "chat"
query_params = {}

For Codex, wire_api only supports chat . If you use the responses mode, you'll need to change to chat . Please also switch model_provider to the newly configured stepfun-chat .

When finishing the configuration, run codex in a new Terminal window to start Codex. Run /status to check the configuration.

/status
📂 Workspace
  • Path: /Users/step-test/
  • Approval Mode: on-request
  • Sandbox: workspace-write
  • AGENTS files: (none)

🧠 Model
  • Name: step-3.5-flash
  • Provider: Stepfun-chat

💻 Client
  • CLI Version: 0.40.0

7.1.5 Use Step 3.5 Flash on Step-DeepResearch (DeepResearch)

1. Use the reference environment setup below and configure MODEL_NAME to Step-3.5-Flash . https://github.com/stepfun-ai/StepDeepResearch?tab=readme-ov-file#1-environment-setup

8. Limitations, Known Issues and Future Directions

1. Token Efficiency . Step 3.5 Flash achieves frontier-level agentic intelligence but currently relies on longer generation trajectories than Gemini 3.0 Pro to reach comparable quality.
2. Efficient Universal Mastery . We aim to unify generalist versatility with deep domain expertise. To achieve this efficiently, we are advancing variants of on-policy distillation, allowing the model to internalize expert behaviors with higher sample efficiency.
3. RL for More Agentic Tasks . While Step 3.5 Flash demonstrates competitive performance on academic agentic benchmarks, the next frontier of agentic AI necessitates the application of RL to intricate, expert-level tasks found in professional work, engineering, and research.
4. Operational Scope and Constraints . Step 3.5 Flash is tailored for coding and work-centric tasks, but may experience reduced stability during distribution shifts. This typically occurs in highly specialized domains or long-horizon, multi-turn dialogues, where the model may exhibit repetitive reasoning, mixed-language outputs, or inconsistencies in time and identity awareness.

9. Co-Developing the Future

We view our roadmap as a living document, evolving continuously based on real-world usage and developer feedback. As we work to shape the future of AGI by expanding broad model capabilities, we want to ensure we are solving the right problems. We invite you to be part of this continuous feedback loop—your insights directly influence our priorities.

• Join the Conversation : Our Discord community is the primary hub for brainstorming future architectures, proposing capabilities, and getting early access updates 🚀
• Report Friction : Encountering limitations? You can open an issue on GitHub or flag it directly in our Discord support channels.

License

This project is open-sourced under the Apache 2.0 License .

1. Introduction

Step3.5 is our most capable open-source reasoning model, purpose-built for agentic workflows. It bridges the gap between massive scale and high performance by combining 196B parameters of knowledge with the inference latency of an 11B model. We prioritized developer needs to balance speed, cost, and accessibility. This enables the creation of production-grade agents that are fast, stable, and cost-effective.

2. Key Capabilities

• Frontier intelligence at 200 tokens/s: Step3.5 matches GPT-5 and Gemini 3.0 Pro in reasoning but runs 4x faster. By leveraging Multi-Token Prediction (MTP-3), Step3.5 predicts three tokens simultaneously, achieving 200 tokens/s for real-time responsiveness.
• Easy local deployment: Despite its massive 196B total parameter count, Step3.5's sparse MoE architecture allows it to run locally on high-end consumer hardware (e.g. Mac Studio M2/M3 Ultra). This enables secure, offline deployment of elite-level intelligence.
• Agentic & coding mastery: Step3.5 is fine-tuned for reliability. It achieves 85.5% on LiveCodeBench and 72.1% on SWE-bench Verified, making it a robust engine for autonomous software engineering and multi-step planning.
• Cost-effective long context: Optimized with a 3:1 sliding window attention strategy (512 window), Step3.5 handles extended contexts with minimal memory overhead, perfect for RAG applications and analyzing large codebases.

3. Benchmarks

Architecture

Key Features:

• Hybrid Attention Schedules and Compensation for SWA

• Mixture-of-Experts Routing And Load balancing

Architecture Details

• Backbone: 45-layer Transformer

• Vocabulary: 128,896 tokens

• Hidden Dim: 4,096

• MoE Blocks:

  • 288 routed experts + 1 shared expert per block
  • Top-8 expert selection per token

• Parameters: Total: 196.81B (Backbone: 196B + MTP Head: 0.81B)

• Activated per token: 11B (excludes embedding/output projections)

• Special Components:

  Multi-token Prediction (MTP) head with sliding-window attention and dense FFN

5. Getting started

Deployment Resource Specifications

• Model Weights: 20 GB
• Runtime Overhead: ~4 GB
• Minimum VRAM Required: 24 GB (e.g., RTX 4090 or A100)

Deploy Step3.5 Locally

For local deployment, Step3.5-preview supports inference frameworks including vLLM and SGLang. Comprehensive deployment instructions are available in the official Github repository.

vLLM and SGLang only support Step3.5-preview on their main branches. you can use their official docker images for inference.

vLLM

Using Docker as:

docker pull vllm/vllm-openai:nightly 

or using pip (must use pypi.org as the index url):

pip install -U vllm --pre --index-url https://pypi.org/simple --extra-index-url https://wheels.vllm.ai/nightly

SGLang

Using Docker as:

docker pull lmsysorg/sglang:dev

or using pip install sglang from source.

transformers

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_PATH = "xxxxxx"
messages = [{"role": "user", "content": "hello"}]
tokenizer = AutoTokenizer.from_pretrained(MODEL_PATH)
inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
)
model = AutoModelForCausalLM.from_pretrained(
    pretrained_model_name_or_path=MODEL_PATH,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
inputs = inputs.to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=128, do_sample=False)
output_text = tokenizer.decode(generated_ids[0][inputs.input_ids.shape[1] :])
print(output_text)

vLLM

vllm serve {xxx} \
     --tensor-parallel-size 4 \
     --speculative-config.method mtp \
     --speculative-config.num_speculative_tokens 1 \
     --tool-call-parser {xxx} \
     --reasoning-parser {xxx} \
     --enable-auto-tool-choice \
     --served-model-name {xxx}

SGLang

python3 -m sglang.launch_server \
  --model-path {xxx} \
  --tp-size 8 \
  --tool-call-parser {xxx}  \
  --reasoning-parser {xxx} \
  --speculative-algorithm EAGLE \
  --speculative-num-steps 3 \
  --speculative-eagle-topk 1 \
  --speculative-num-draft-tokens 4 \
  --mem-fraction-static 0.8 \
  --served-model-name {xxx} \
  --host 0.0.0.0 \
  --port 8000

Parameter Instructions

• When using vLLM and SGLang , thinking mode is enabled by default when sending requests.
• Both support tool calling. Please use OpenAI-style tool description format for calls.

📄 License

This project is open-sourced under the Apache 2.0 License .