nex-agi / Nex-N2-Mini-NVFP4

---

Nex-N2-Mini-NVFP4

NVFP4 (4-bit) quantized build of Nex-N2-mini — an agentic model with Agentic Thinking.

This repository contains a post-training-quantized checkpoint of nex-agi/Nex-N2-mini , produced with llm-compressor and stored in the compressed-tensors format. The MoE expert stack is quantized to NVFP4 (W4A4) and the attention / output projections to FP8 (W8A8), shrinking the checkpoint from 70.2 GB to 24.8 GB (~2.8×) so the model fits comfortably on a single GPU.

---

Quantization Details

The model is quantized with compressed-tensors ( quant_method: compressed-tensors , format: mixed-precision ). The full recipe ships in recipe.yaml .

NVFP4 — W4A4 ( nvfp4-pack-quantized )

Applied to the MoE feed-forward stack in layers 0–31 — this is the bulk of the parameters:

• model.language_model.layers.{0..31}.mlp.experts.{0..255}.{gate,up,down}_proj
• model.language_model.layers.{0..31}.mlp.shared_expert.{gate,up,down}_proj

FP8 — W8A8 ( float-quantized )

Applied to attention, the linear-attention projections, the LM head, and the MoE stack of the last 8 layers (32–39) , which are the most quantization-sensitive:

• self_attn.{q,k,v,o}_proj
• linear_attn.{in_proj_qkv,in_proj_z,out_proj}
• lm_head
• layers.{32..39}.mlp.experts.*.{gate,up,down}_proj and layers.{32..39}.mlp.shared_expert.{gate,up,down}_proj

Kept in BF16

• The full vision tower ( model.visual.* , 333 tensors)
• MoE routers — mlp.gate , mlp.shared_expert_gate
• Linear-attention gating / state projections — linear_attn.in_proj_a , linear_attn.in_proj_b , linear_attn.conv1d , A_log , dt_bias , and all linear_attn.norm*
• Token embeddings and all RMSNorm weights

The KV cache is not quantized ( kv_cache_scheme: null ).

---

Hardware

NVFP4 reaches its full speedup on NVIDIA Blackwell GPUs (SM100 / SM120 — B200, GB200, RTX PRO 6000 Blackwell), which have native FP4 tensor cores. vLLM can still load and run this checkpoint on pre-Blackwell hardware through its NVFP4 dequantization/emulation path, in which case you get the memory savings but not the throughput gain.

At 24.8 GB of weights the model serves on a single 80 GB card with room for a long-context KV cache; the launch command below is tuned for that shape.

---

Usage

Deployment with vLLM

Note: Qwen3.5-MoE support, the NVFP4 compressed-tensors kernels, and the --language-model-only flag all require a recent vLLM build. Use a current release or nightly.

vllm serve /models/Nex-N2-Mini-NVFP4 \
  --served-model-name nex-n2 \
  --host 0.0.0.0 --port 8000 \
  --trust-remote-code --language-model-only \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 32 \
  --gpu-memory-utilization 0.60 \
  --max-num-batched-tokens 8192 \
  --enable-chunked-prefill

Flag notes:

• --language-model-only skips the (unquantized, BF16) vision tower and serves the model as a pure text LLM. Drop it if you want the multimodal path.
• --max-model-len 131072 is a conservative cap; the checkpoint's max_position_embeddings is 262144 .
• --gpu-memory-utilization 0.60 and --max-num-seqs 32 leave generous headroom — raise them if you have spare VRAM and want more concurrency.
• --tensor-parallel-size 1 is sufficient at this precision. Increase it for extra KV-cache capacity, not for weight capacity.

Query it through the OpenAI-compatible API:

curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nex-n2",
    "messages": [{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
    "temperature": 0.7,
    "top_p": 0.95,
    "top_k": 40
  }'

Function Calling and Reasoning Parser

Nex-series models emit explicit reasoning traces and support robust function calling. To have vLLM split the reasoning trace from the final answer and parse tool calls into the OpenAI tool_calls schema, add:

  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Without these flags the reasoning trace and the <tool_call> blocks are returned inline in content , and you parse them yourself.

Recommended Sampling Parameters

• temperature : 0.7
• top_p : 0.95
• top_k : 40

Deployment with SGLang

The base model is best served with the customized sglang fork described in the Nex-N2-mini card . That fork does not currently consume compressed-tensors NVFP4 checkpoints — use the BF16 base model there, and this repository with vLLM.

---

Performance

The scores below are the published results for the BF16 base models and are reproduced here for reference. They were not re-measured on this NVFP4 checkpoint; expect a small degradation typical of 4-bit post-training quantization. If you benchmark it, we would welcome the numbers in the Community tab.

---

About Nex-N2

Nex-N2 is an agent model built for real-world productivity scenarios. With first-tier coding and agentic capabilities, it keeps driving complex, long-horizon tasks forward in real environments to deliver stable, end-to-end results.

Rather than treating reasoning, tool use, and environment execution as separate capabilities, Nex-N2 unifies them through an Agentic Thinking framework that connects requirement understanding, task planning, code implementation, environmental feedback, evaluation and debugging, and continuous iteration into a single closed loop. The framework has two parts:

• Adaptive Thinking lets the model decide on its own when to think and how deeply — executing simple actions quickly while reasoning thoroughly on critical decisions.
• Coherent Thinking carries one consistent reasoning paradigm across general reasoning and diverse agentic tasks, staying consistent across tasks and modalities to enable stable capability transfer.

Nex-N2 ships in two variants, both post-trained on the Qwen3.5 series: Nex-N2-Pro (built on Qwen3.5-397B-A17B ) and Nex-N2-mini (built on Qwen3.5-35B-A3B-Base ).

Model Architecture

Related Models

• Nex-N2-Pro: Hugging Face | ModelScope
• Nex-N2-mini (BF16 base): Hugging Face | ModelScope

License

Released under the Apache 2.0 license, matching the base model.