CQSystems / Google-Gemma-4-E4B-It-GGUF

CQ-Gemma-E4B-IT (GGUF)

This repository contains quantized GGUF formats of the Google Gemma 4 E4B Instruction-Tuned model, published by CQ Systems .

These files are optimized for local inference on consumer hardware, particularly Apple Silicon (M-series Macs) and standard GPUs, using tools like llama.cpp , LM Studio, and Ollama.

Available Files

We provide three different versions of the model to suit different memory and performance requirements.

How to Use

Using llama.cpp

You can run this model directly in your terminal using the compiled llama.cpp CLI tool.

# Basic chat launch with the recommended Q4_K model
./llama-cli -m CQ_Gemma4_E4B_f16_Q4_K.gguf \
-c 8192 \
-n 1024 \
-p "You are a helpful assistant.\n\nUser: Write a short joke about saving RAM.\nAssistant:"

Using LM Studio / Ollama

1. Download the .gguf file of your choice (we recommend CQ_Gemma4_E4B_f16_Q4_K.gguf).
2. LM Studio: Drag and drop the downloaded file into your LM Studio model folder, or use the local import feature.
3. Ollama: Create a Modelfile with the line FROM ./CQ_Gemma4_E4B_f16_Q4_K.gguf, then run ollama create cq-gemma-e4b -f Modelfile.

About the Base Model (Gemma 4 E4B)

Gemma 4 is a family of open, multimodal models built by Google DeepMind. The E4B variant is explicitly optimized for on-device processing, making it incredibly powerful for local deployment on laptops and mobile devices.

Key Capabilities

• Thinking Mode: Features a built-in reasoning mode that lets the model think step-by-step before answering.
• Massive Context: Supports a context window of up to 128K tokens .
• Parameter Efficiency: "E4B" stands for 4.5B effective parameters (8B total with Per-Layer Embeddings). This allows it to perform well above its weight class while keeping active memory lookups fast.
• System Prompts: Gemma 4 uses standard system, assistant, and user roles natively.

(Note: While the base Gemma 4 E4B supports audio and image inputs natively, these GGUF files primarily support text generation out-of-the-box via llama.cpp. Multimodal capabilities in GGUF require an accompanying mmproj vision/audio encoder file).

Thinking Mode Configuration

To enable the built-in reasoning capabilities, include the <|think|> token at the start of your system prompt. The model will then output its internal reasoning inside <|channel>thought\n ... <channel|> tags before providing the final response.

License & Acknowledgements

• Base Model: Google DeepMind
• License: Gemma 4 License
• Quantization: CQ Systems using llama.cpp

For full benchmark details, evaluations, and ethical considerations, please refer to the original Google Gemma 4 E4B documentation .