ababb123 / Gemma4-26B-A4B-Balanced

Inference parameters:

• temperature=1.0, top_p=0.95, top_k=64

Important:

• Use --jinja with llama.cpp for proper chat template handling
• Vision support requires the mmproj file alongside the main GGUF. Place images before text in your prompt for best vision performance.
• Keep at least 32K context for serious agentic work; the model can take much more (256K native) if you need it
• Sliding window is baked into the architecture — no special flag needed

Turning Thinking On/Off

Gemma4 has thinking mode controlled via enable_thinking in the chat template. It's the same pattern as Qwen3.6 — set false for faster, shorter replies and true (default) when you want chain-of-thought.

LM Studio

1. Load the model
2. Right-side settings panel → Model Settings → Prompt Template (or Chat Template Options )
3. Set enable_thinking to false (or true ) in the template kwargs

llama.cpp

llama-server — set as default for all requests:

llama-server -m Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf \
  --mmproj mmproj-Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-f16.gguf \
  --jinja -c 32768 -ngl 99 \
  --chat-template-kwargs '{"enable_thinking": false}'

Per-request via the OpenAI-compatible API:

{
  "model": "gemma4-26b-a4b",
  "messages": [{"role": "user", "content": "..."}],
  "chat_template_kwargs": {"enable_thinking": false}
}

Usage

Works with llama.cpp, LM Studio, Jan, koboldcpp, and other GGUF-compatible runtimes.

llama-server:

llama-server -m Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf \
  --mmproj mmproj-Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-f16.gguf \
  --jinja -c 32768 -ngl 99

llama-cli:

llama-cli -m Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-Q4_K_P.gguf \
  --mmproj mmproj-Gemma4-26B-A4B-Uncensored-HauhauCS-Balanced-f16.gguf \
  --jinja -c 32768 -ngl 99