These files were quantized using
llama.cpp
to provide various compressed versions of the model for local inference on lower-VRAM hardware.
Available Quantizations
The following quantization formats are provided to allow you to balance between memory usage, speed, and quality:
Q8_0:
8-bit quantization. Very close to the original F16 model in quality, but requires the most memory.
Q6_K:
6-bit quantization. Excellent balance of quality and size.
Q5_K_M:
5-bit quantization. Good middle ground for lower-end hardware while retaining strong coherence.
Q4_K_M:
4-bit quantization. The recommended standard for everyday local use. High speed, low memory, minor perplexity hit.
Q4_K_S:
4-bit quantization (small). Slightly smaller and faster than Q4_K_M, with a minor drop in accuracy.
Q3_K_M:
3-bit quantization. Extreme compression for very limited hardware. Noticeable degradation in complex reasoning, but suitable for basic text generation.
How to Use
You can run these GGUF models using any UI or terminal tool that supports
llama.cpp
, such as:
If you have
llama.cpp
compiled locally, you can run the model directly from the terminal.
# Example using the Q4_K_M quant
./llama-cli -m supergemma4-Q4_K_M.gguf -p "You are a helpful assistant. How do I write a Python script?" -n 512 -c 2048
Runs of Abiray supergemma4-e4b-abliterated-GGUF on huggingface.co
323
Total runs
22
24-hour runs
78
3-day runs
96
7-day runs
49
30-day runs
More Information About supergemma4-e4b-abliterated-GGUF huggingface.co Model
supergemma4-e4b-abliterated-GGUF huggingface.co
supergemma4-e4b-abliterated-GGUF huggingface.co is an AI model on huggingface.co that provides supergemma4-e4b-abliterated-GGUF's model effect (), which can be used instantly with this Abiray supergemma4-e4b-abliterated-GGUF model. huggingface.co supports a free trial of the supergemma4-e4b-abliterated-GGUF model, and also provides paid use of the supergemma4-e4b-abliterated-GGUF. Support call supergemma4-e4b-abliterated-GGUF model through api, including Node.js, Python, http.
supergemma4-e4b-abliterated-GGUF huggingface.co is an online trial and call api platform, which integrates supergemma4-e4b-abliterated-GGUF's modeling effects, including api services, and provides a free online trial of supergemma4-e4b-abliterated-GGUF, you can try supergemma4-e4b-abliterated-GGUF online for free by clicking the link below.
Abiray supergemma4-e4b-abliterated-GGUF online free url in huggingface.co:
supergemma4-e4b-abliterated-GGUF is an open source model from GitHub that offers a free installation service, and any user can find supergemma4-e4b-abliterated-GGUF on GitHub to install. At the same time, huggingface.co provides the effect of supergemma4-e4b-abliterated-GGUF install, users can directly use supergemma4-e4b-abliterated-GGUF installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
supergemma4-e4b-abliterated-GGUF install url in huggingface.co: