caiovicentino1 / Gemma-4-E4B-it-PolarQuant-Multi

🧊 Gemma-4-E4B-it — PolarQuant Multi (PQ1-PQ8)

ALL quantization variants of Google's hottest model (108K downloads in 2 days).

Multimodal: text + image + audio + video | Apache 2.0 | 128K context | Edge-optimized

📊 All Variants

Quality scale:
PQ1: ██░░░░░░░░ 0.788   PQ4: █████████░ 0.991
PQ2: █████████░ 0.931   PQ5: ██████████ 0.996 ← recommended
PQ3: █████████░ 0.977   PQ8: ██████████ 0.999

✅ Generation Quality (Verified)

Real generation tests comparing PolarQuant output vs the original BF16 model on 3 prompts:

Summary

Recommendation: Use PQ5 (11.7 GB) for the best quality-to-size ratio. It produces outputs identical to the original BF16 model across all tested prompts.

🚀 Quick Start

import torch
from transformers import AutoTokenizer
from safetensors.torch import load_file

# Choose your variant: PQ1, PQ2, PQ3, PQ4, PQ5, PQ6, PQ8
VARIANT = "PQ5"
REPO_ID = "caiovicentino1/Gemma-4-E4B-it-PolarQuant-Multi"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(REPO_ID, trust_remote_code=True)

# Load quantized weights
from huggingface_hub import hf_hub_download

codes_path = hf_hub_download(REPO_ID, f"{VARIANT}/codes.safetensors")
bf16_path = hf_hub_download(REPO_ID, f"{VARIANT}/bf16.safetensors")

codes = load_file(codes_path)
bf16_weights = load_file(bf16_path)

print(f"Loaded {VARIANT}: {len(codes)} quantized layers + {len(bf16_weights)} BF16 layers")

Full Inference with PolarQuant

# pip install polarquant
from polarquant import PolarQuantModel

model = PolarQuantModel.from_pretrained(
    "caiovicentino1/Gemma-4-E4B-it-PolarQuant-Multi",
    variant="PQ5",          # Choose: PQ1-PQ8
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

tokenizer = AutoTokenizer.from_pretrained(
    "caiovicentino1/Gemma-4-E4B-it-PolarQuant-Multi",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "What is 2+2?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)

with torch.no_grad():
    output = model.generate(inputs, max_new_tokens=128)
    
print(tokenizer.decode(output[0], skip_special_tokens=True))

💻 Hardware Compatibility

🏗️ Architecture

Gemma-4-E4B-it (4.5B effective, 8B with PLE embeddings)
├── 42 layers, hybrid sliding window (512) + global attention
├── Multimodal: text + image (~150M) + audio (~300M) + video
├── 128K context, 262K vocabulary
├── Per-Layer Embeddings (PLE) for edge efficiency
├── Function calling + reasoning mode
└── 386 layers quantized, rest preserved in BF16

🔬 Why PolarQuant > GGUF

Lloyd-Max centroids are mathematically optimal for Gaussian-distributed weights:

PQ3 (cos_sim 0.977) > GGUF Q3_K_M (~0.95)  — same size, better quality
PQ5 (cos_sim 0.996) > GGUF Q5_K_M (~0.99)  — same size, better quality

GGUF uses uniform spacing between quantization levels — suboptimal. PolarQuant places centroids where the weight density is highest (Lloyd-Max algorithm). Hadamard rotation spreads outliers before quantizing — each code carries maximum information.

📦 Files

PQ1/codes.safetensors + bf16.safetensors   (1-bit, 10.1 GB)
PQ2/codes.safetensors + bf16.safetensors   (2-bit, 10.5 GB)
PQ3/codes.safetensors + bf16.safetensors   (3-bit, 10.9 GB)
PQ4/codes.safetensors + bf16.safetensors   (4-bit, 11.3 GB)
PQ5/codes.safetensors + bf16.safetensors   (5-bit, 11.7 GB) ← recommended
PQ6/codes.safetensors + bf16.safetensors   (6-bit, 12.1 GB)
PQ8/codes.safetensors + bf16.safetensors   (8-bit, 12.9 GB)
config.json + tokenizer files

🔗 Links

• 📄 Paper — arXiv:2603.29078
• 💻 GitHub — PolarEngine
• 📦 PyPI — pip install polarquant
• 🏠 Original Model

📖 Citation

@article{polarquant2026,
  title={PolarQuant: Hadamard-Rotated Lloyd-Max Quantization for Large Language Models},
  author={Vicentino, Caio},
  journal={arXiv preprint arXiv:2603.29078},
  year={2026}
}

Quantized with PolarQuant — Hadamard + Lloyd-Max optimal quantization. First multi-variant PolarQuant release.

🧪 PQ2-Mixed: Smallest Functional Variant

MLP in PQ2 (4 centroids) + Attention in PQ4 (16 centroids) — smaller than PQ3 pure, better quality than PQ2 pure.

Generation Test

PQ2-Mixed is the smallest variant that produces correct factual answers. Use PQ3+ for production.