AtomicChat / gemma-4-26B-A4B-it-assistant-GGUF

huggingface.co
Total runs: 2.3K
24-hour runs: -227
7-day runs: -610
30-day runs: -920
Model's Last Updated: July 24 2026
text-generation

Introduction of gemma-4-26B-A4B-it-assistant-GGUF

Model Details of gemma-4-26B-A4B-it-assistant-GGUF

Gemma 4 26B-A4B Assistant — GGUF (Atomic Chat)

GGUF builds of google/gemma-4-26B-A4B-it-assistant — the official Gemma 4 Multi-Token Prediction (MTP) drafter for google/gemma-4-26B-A4B-it . Use it as a speculative-decoding draft model alongside the matching Gemma 4 target to get a meaningful decoding speedup at zero quality loss.

Approximate size: 0.4B (assistant) / 27B MoE target .

These GGUFs use the custom gemma4_assistant architecture and will not load in stock llama.cpp . They require the atomic-llama-cpp-turboquant fork, which adds:

  • the gemma4_assistant MTP drafter arch (incl. the centroid LM head for E2B/E4B),
  • TurboQuant KV-cache quantization ( -ctk turbo3 -ctv turbo3 ),
  • the --mtp-head / --spec-type mtp runtime flags.

Loading these files in upstream ggml-org/llama.cpp will fail with an unknown architecture error.

Files
File Quant Size Notes
gemma-4-26B-A4B-it-assistant.F16.gguf F16 815.6 MB reference (smallest quality loss vs source)
gemma-4-26B-A4B-it-assistant.Q8_0.gguf Q8_0 440.4 MB near-lossless 8-bit
gemma-4-26B-A4B-it-assistant.Q5_K_M.gguf Q5_K_M 326.4 MB balanced k-quant
gemma-4-26B-A4B-it-assistant.Q4_K_M.gguf Q4_K_M 310.4 MB recommended default for speculative-decoding draft
gemma-4-26B-A4B-it-assistant.Q4_K_S.gguf Q4_K_S 306.3 MB smallest k-quant
Quick start

Build the fork:

git clone https://github.com/AtomicBot-ai/atomic-llama-cpp-turboquant
cd atomic-llama-cpp-turboquant
# Pick one of the platform-specific configurations:
cmake -B build -DGGML_METAL=ON          # Apple Silicon
# cmake -B build -DGGML_CUDA=ON         # NVIDIA
# cmake -B build                        # CPU-only
cmake --build build --target llama-server llama-cli llama-quantize -j

Download the assistant drafter (this repo) and the matching Gemma 4 target:

hf download AtomicChat/gemma-4-26B-A4B-it-assistant-GGUF \
    --include "*Q4_K_M.gguf" --local-dir ./models
# Any GGUF build of the matching target model works; e.g. unsloth's:
hf download unsloth/gemma-4-26B-A4B-it-GGUF \
    --include "*Q4_K_M*.gguf" --local-dir ./models

Run llama-server with MTP speculative decoding + TurboQuant KV cache:

./build/bin/llama-server \
    -m         ./models/gemma-4-26B-A4B-it-Q4_K_M.gguf \
    --mtp-head ./models/gemma-4-26B-A4B-it-assistant.Q4_K_M.gguf \
    --spec-type mtp \
    --draft-block-size 3 --draft-max 8 --draft-min 0 \
    -ngl 99 -ngld 99 \
    -ctk turbo3 -ctv turbo3 -ctkd turbo3 -ctvd turbo3 \
    -fa on -c 16384 --host 127.0.0.1 --port 8080

A ready-made launcher lives at scripts/run-gemma4-26b-a4b-mtp-server.sh in the fork ( MTP_PRESET=throughput|lift|balanced|quality ).

How MTP works here

Gemma 4 ships with a small "assistant" head that predicts several future tokens from the target model's last hidden state. In atomic-llama-cpp-turboquant it is loaded as a separate GGUF via --mtp-head and drives a custom speculative decoder (block_size 3, draft_max 16 for quality preset). The verifier runs the target model in parallel, guaranteeing the same output distribution as plain greedy/sampled decoding.

TurboQuant KV cache

turbo3 is the KV-cache quantization scheme used in this fork; it significantly reduces KV memory and bandwidth at long contexts with no measurable quality regression on Gemma 4. Apply it to both target and drafter via -ctk turbo3 -ctv turbo3 -ctkd turbo3 -ctvd turbo3 .

License & attribution

Released under the Gemma Terms of Use .

Acknowledgements
  • Google DeepMind — Gemma 4 family and the MTP drafters.
  • ggml-org/llama.cpp — upstream inference engine.
  • TurboQuant primitives — KV-cache quantization scheme integrated in the fork.

Atomic Chat

Runs of AtomicChat gemma-4-26B-A4B-it-assistant-GGUF on huggingface.co

2.3K
Total runs
-227
24-hour runs
-287
3-day runs
-610
7-day runs
-920
30-day runs

More Information About gemma-4-26B-A4B-it-assistant-GGUF huggingface.co Model

More gemma-4-26B-A4B-it-assistant-GGUF license Visit here:

https://choosealicense.com/licenses/apache-2.0

gemma-4-26B-A4B-it-assistant-GGUF huggingface.co

gemma-4-26B-A4B-it-assistant-GGUF huggingface.co is an AI model on huggingface.co that provides gemma-4-26B-A4B-it-assistant-GGUF's model effect (), which can be used instantly with this AtomicChat gemma-4-26B-A4B-it-assistant-GGUF model. huggingface.co supports a free trial of the gemma-4-26B-A4B-it-assistant-GGUF model, and also provides paid use of the gemma-4-26B-A4B-it-assistant-GGUF. Support call gemma-4-26B-A4B-it-assistant-GGUF model through api, including Node.js, Python, http.

gemma-4-26B-A4B-it-assistant-GGUF huggingface.co Url

https://huggingface.co/AtomicChat/gemma-4-26B-A4B-it-assistant-GGUF

AtomicChat gemma-4-26B-A4B-it-assistant-GGUF online free

gemma-4-26B-A4B-it-assistant-GGUF huggingface.co is an online trial and call api platform, which integrates gemma-4-26B-A4B-it-assistant-GGUF's modeling effects, including api services, and provides a free online trial of gemma-4-26B-A4B-it-assistant-GGUF, you can try gemma-4-26B-A4B-it-assistant-GGUF online for free by clicking the link below.

AtomicChat gemma-4-26B-A4B-it-assistant-GGUF online free url in huggingface.co:

https://huggingface.co/AtomicChat/gemma-4-26B-A4B-it-assistant-GGUF

gemma-4-26B-A4B-it-assistant-GGUF install

gemma-4-26B-A4B-it-assistant-GGUF is an open source model from GitHub that offers a free installation service, and any user can find gemma-4-26B-A4B-it-assistant-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of gemma-4-26B-A4B-it-assistant-GGUF install, users can directly use gemma-4-26B-A4B-it-assistant-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

gemma-4-26B-A4B-it-assistant-GGUF install url in huggingface.co:

https://huggingface.co/AtomicChat/gemma-4-26B-A4B-it-assistant-GGUF

Url of gemma-4-26B-A4B-it-assistant-GGUF

gemma-4-26B-A4B-it-assistant-GGUF huggingface.co Url

Provider of gemma-4-26B-A4B-it-assistant-GGUF huggingface.co

AtomicChat
ORGANIZATIONS

Other API from AtomicChat

huggingface.co

Total runs: 339
Run Growth: -949
Growth Rate: -279.94%
Updated:July 23 2026
huggingface.co

Total runs: 32
Run Growth: -37
Growth Rate: -115.63%
Updated:July 28 2026