This is an int4 quantized version of
google/gemma-4-E4B-it
using
SDNQ
(SD.Next Quantization) with the dynamic option, Hadamard Rotation and quantized embeddings.
Note: You need SDNQ v0.2.1 or superior (the quantized embeddings + Hadamard path requires the
nn.Embedding
fix that landed in 0.2.1).
Usage
Load it with
transformers
like the original model. You need to
import sdnq
first so it registers into the transformers quantizer registry — without it, loading fails with a size-mismatch error:
import sdnq # registers the SDNQ quantizer into transformersimport torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "OzzyGT/gemma_4_E4B_it_sdnq_dynamic_4bit"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="cuda")
messages = [{"role": "user", "content": [{"type": "text", "text": "Explain quantization in one sentence."}]}]
inputs = processor.apply_chat_template(
messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt"
).to(model.device)
out = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Runs of OzzyGT gemma_4_E4B_it_sdnq_dynamic_4bit on huggingface.co
10
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
4
30-day runs
More Information About gemma_4_E4B_it_sdnq_dynamic_4bit huggingface.co Model
More gemma_4_E4B_it_sdnq_dynamic_4bit license Visit here:
gemma_4_E4B_it_sdnq_dynamic_4bit huggingface.co is an AI model on huggingface.co that provides gemma_4_E4B_it_sdnq_dynamic_4bit's model effect (), which can be used instantly with this OzzyGT gemma_4_E4B_it_sdnq_dynamic_4bit model. huggingface.co supports a free trial of the gemma_4_E4B_it_sdnq_dynamic_4bit model, and also provides paid use of the gemma_4_E4B_it_sdnq_dynamic_4bit. Support call gemma_4_E4B_it_sdnq_dynamic_4bit model through api, including Node.js, Python, http.
gemma_4_E4B_it_sdnq_dynamic_4bit huggingface.co is an online trial and call api platform, which integrates gemma_4_E4B_it_sdnq_dynamic_4bit's modeling effects, including api services, and provides a free online trial of gemma_4_E4B_it_sdnq_dynamic_4bit, you can try gemma_4_E4B_it_sdnq_dynamic_4bit online for free by clicking the link below.
OzzyGT gemma_4_E4B_it_sdnq_dynamic_4bit online free url in huggingface.co:
gemma_4_E4B_it_sdnq_dynamic_4bit is an open source model from GitHub that offers a free installation service, and any user can find gemma_4_E4B_it_sdnq_dynamic_4bit on GitHub to install. At the same time, huggingface.co provides the effect of gemma_4_E4B_it_sdnq_dynamic_4bit install, users can directly use gemma_4_E4B_it_sdnq_dynamic_4bit installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
gemma_4_E4B_it_sdnq_dynamic_4bit install url in huggingface.co: