CohereLabs / North-Mini-Code-1.0-w4a16

Model Card for North Mini Code

Model Summary

North Mini Code is an open weights research release of a 30B-A3B parameter model optimized for code generation, agentic software engineering, and terminal tasks.

Developed by: Cohere and Cohere Labs

• Point of Contact: Cohere Labs
• License: Apache 2.0
• Model: North Mini Code
• Model Size: 30B total; 3B active
• Context length: 256K & 64K max output
• Quantization: NVFP4 W4A16

For more details about this model, please check out our blog post .

Try North Mini Code

You can try out North Mini Code before downloading the weights in OpenCode and our hosted Hugging Face Space .

Evaluation

Usage

To use our model in transformers, please use our BF16 model weights. Our NVFP4_W4A16 checkpoint is designed to be used with vLLM and MLX-VLM and is not compatible with transformers due to lack of native 4-bit support.

vLLM

You can run the model in vLLM. Please use vLLM main for North Mini Code until a new release is available, and accurate response parsing also requires installing Cohere’s melody library.

uv pip install "git+https://github.com/vllm-project/vllm.git"
uv pip install cohere_melody>=0.9.0

Then the vLLM server can be started with the following command:

vllm serve CohereLabs/North-Mini-Code-1.0-w4a16 \
  -tp 1 \
  --max-model-len 320000 \
  --tool-call-parser cohere_command4 \
  --reasoning-parser cohere_command4 \
  --enable-auto-tool-choice

Use locally deployed North Mini Code in OpenCode:

Please use OpenCode > v1.17.0.

brew install anomalyco/tap/opencode

To use locally deployed North Mini Code in Opencode, please use this config which enables interleaved reasoning:

{
  "$schema": "https://opencode.ai/config.json",
  "model": "vllm/CohereLabs/North-Mini-Code-1.0-w4a16",
  "provider": {
    "vllm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Local vLLM server",
      "options": {
        "baseURL": "http://127.0.0.1:8000/v1",
        "apiKey": "EMPTY"
      },
      "models": {
        "CohereLabs/North-Mini-Code-1.0-w4a16": {
          "name": "North-Mini-Code-1.0",
          "interleaved": {
            "field": "reasoning"
          },
          "limit": {
            "context": 256000,
            "output": 64000
          }
        }
      }
    }
  }
}

MLX-VLM

You can also run the model in MLX-VLM. Please use main for North Mini Code until a new release is available.

uv pip install "git+https://github.com/Blaizzy/mlx-vlm.git@main"

Then the mlx_vlm server can be started with the following command:

mlx_vlm.server \
  --model CohereLabs/North-Mini-Code-1.0-w4a16 \
  --enable-thinking \
  --thinking-start-token "<|START_THINKING|>" \
  --thinking-end-token "<|END_THINKING|>"

Opencode config:

Actual limit depends on your device

{
  "$schema": "https://opencode.ai/config.json",
  "model": "mlx-vlm/CohereLabs/North-Mini-Code-1.0-w4a16",
  "provider": {
    "mlx-vlm": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "MLX VLM Local",
      "options": {
        "baseURL": "http://127.0.0.1:8080/v1",
        "apiKey": "EMPTY"
      },
      "models": {
        "CohereLabs/North-Mini-Code-1.0-w4a16": {
          "name": "North-Mini-Code-1.0",
          "interleaved": {
            "field": "reasoning"
          },
          "limit": {
            "context": 256000,
            "output": 64000
          }
        }
      }
    }
  }
}

Model Details

Input : Text only.

Output : Model generates text.

Model Architecture : North-Mini-Code-1.0 is a decoder-only Transformer-based sparse Mixture-of-Experts model. It uses an efficient attention implementation, interleaved between sliding-window attention with RoPE and global attention with no positional embeddings, in a 3:1 ratio. The feed-forward block is an MoE block with 128 experts, of which 8 are activated per token. Each expert block is an FFN block with SwiGLU activation. The router applies a sigmoid activation function to the logits before the top-k selection. We also use a single dense layer before the sparse layers. North-Mini-Code-1.0 was post-trained using a two-stage cascaded supervised fine-tuning (SFT) followed by reinforcement learning with verifiable rewards (RLVR), focusing on agentic coding. For more technical details, please check out our blog post .

Quantization Methodology: We use NVFP4 W4A16 quantization (4-bit weights, 16-bit activations) for this model, delivering a much smaller memory footprint (~ 18-20GB) and faster inference while preserving coding accuracy. We quantize the MoE experts only, keeping attention, the dense layer, and the router at higher precision. Since the experts hold most of the model's parameters, this captures the bulk of the savings with minimal quality loss. To preserve quality, we use Quantization-Aware Distillation (QAD), training the quantized model to match the unquantized model's outputs, achieving >99% overall accuracy recovery across our evaluations. Since only weights are quantized, this format does not require native FP4 hardware and runs on pre-Blackwell GPUs such as Hopper and Ada.

Context Length: North-Mini-Code-1.0 supports a context length of 256K & 64K output length.

Model Card Contact

For errors or additional questions about details in this model card, contact [ [email protected] ].