ByteOtter / gemma-4-E4B-it-CADA-IQ2_XXS

base_model: unsloth/gemma-4-E4B-it-GGUF language: - multilingual pipeline_tag: text-generation tags: - gemma4 - gguf - llama.cpp - mixed-precision - task-aware-quantization - reasoning

Gemma 4 E4B IT — reasoning-directed mixed-precision IQ2_XXS GGUF

This is an experimental, reasoning-directed mixed-precision GGUF derived from the BF16 language-model artifact in unsloth/gemma-4-E4B-it-GGUF . It was produced by QLAB from Google's instruction-tuned Gemma 4 E4B weights and independently evaluated by MLAB against a stock IQ2_XXS quantization made from the same source and the same reasoning calibration matrix.

The main result is that the allocation recovers nearly all of the reasoning capability lost to 2-bit quantization. On MLAB's independent 128-item reasoning suite, this allocation scored 69.531%, compared with 28.906% for the byte-matched stock IQ2_XXS comparator. The paired allocation effect was +40.625 percentage points with a 95% bootstrap interval of +24.638 to +55.435 points.

Against the BF16 source the difference was -2.344 points, with a 95% interval of -12.121 to +8.974 that spans zero — this quantization is not statistically distinguishable from its 14.02 GiB source on that suite, at 23.68% of the size. That is a measurement on one suite of one corpus, not a claim of general BF16 parity. Reasoning is the only category this allocation was directed at, and several other categories remain well below source. See Interpretation and limitations .

Model details

The model preserves the source tokenizer, the embedded Gemma chat template, and the recorded base-model provenance ( google/gemma-4-E4B-it ).

Tensor allocation

This is not a uniform IQ2_XXS quantization. QLAB assigned precision per tensor under a matched-size budget. The resulting file is 0.4538% smaller than the stock IQ2_XXS comparator it is measured against.

Usage

Use a recent build of llama.cpp:

llama-cli \
  -m gemma-4-E4B-it-CADA-IQ2_XXS.gguf \
  -cnv

Or start an OpenAI-compatible server:

llama-server \
  -m gemma-4-E4B-it-CADA-IQ2_XXS.gguf \
  --host 127.0.0.1 \
  --port 8080

The GGUF contains the language model only. A multimodal projector is not included, and image- or audio-input behavior was not evaluated for this release.

Independent MLAB evaluation

MLAB evaluated the exact published candidate SHA-256 shown above. The comparator was a stock IQ2_XXS artifact produced from the same BF16 source with the same reasoning-category importance matrix.

Target-suite result

The candidate and stock comparator differ by a 0.4538% symmetric byte delta, inside the prespecified ±0.5% comparison window.

Paired candidate-minus-stock reasoning effect:

• Estimate: +40.625 percentage points
• 95% bootstrap interval: +24.638 to +55.435 points
• Paired items: 128
• Capability flow: 50 source-only items recovered, 2 items that both source and stock answered correctly broken, 6 items recovered that both source and stock missed, and 2 stock-only items broken; net gain over stock: 52 items

Relative to the BF16 source, the candidate's measured reasoning score was 2.344 points lower (95% interval -12.121 to +8.974, spanning zero) and its point-estimate retention was 96.74% .

Other measured suites

These are secondary diagnostics, not additional optimization targets. Deltas are paired candidate-minus-stock results.

All artifacts were measured on the full item count for every suite.

Runtime snapshot

MLAB ran eight position-balanced runtime trials with full GPU offload, flash attention, 32 CPU threads, batch size 512, and ubatch size 128. On that test system:

These numbers are system- and prompt-dependent, and each trial generated only a few tokens, so they measure latency more than sustained throughput. Peak VRAM was not measured. Across the full 3,840-cell evaluation workload the candidate sustained 63.0 tok/s against the source's 27.4 tok/s.

Evaluation protocol

• Evaluation pack: mlab-v1
• MLAB run: v291-gemma4-e4b-qat-reasoning-iq2_xxs-r1
• Bootstrap resamples: 10,000
• Bootstrap seed: 5569898590874107905
• MLAB report SHA-256: f865ca9e189d0505a41897f0accd7c170694f87c4e8bcef70639d07f4e5d5f35
• llama.cpp commit: 2fc24394dcfb33dd3fa5e15207373e66666130cf

The ordinary stock/no-imatrix lane does not exist at this rung. IQ2_XXS tensors cannot be materialized without importance-matrix data, so that artifact cannot legally be built and MLAB typed the lane imatrix_required . As a result MLAB did not measure the full allocation-plus-imatrix product effect or isolate the imatrix-only effect. The reported matched comparison isolates the allocation effect relative to stock quantization using the same category imatrix.

Interpretation and limitations

The allocation passed QLAB's readiness gate and outscored all ten independently constructed byte-matched controls on held-out rows, by margins from +3.498 to +15.807 macro points. That is evidence the result is not an artifact of the search procedure. It is a single run on a single model, and it has not been replicated.

Known weaknesses of this artifact:

• Coding is degraded and should not be relied on. 36 of 128 coding items never terminated within a 1,024-token budget. The code it emits is structurally sound but verbose. Measured score is 24.219% against the source's 41.406%.
• Structured output retains 55.81% of source and math retains 60.61% .
• Stability retains 73.79% of source , and is the one suite where this allocation scores slightly below the stock comparator.
• Reasoning parity with BF16 is a suite-level result on 128 held-out items. The interval is wide, and it does not transfer to categories the allocation was not directed at.
• Multimodal behavior is untested.

Intended use

This artifact is intended for research and experimentation with aggressive, task-directed GGUF quantization, especially where a roughly 3.3 GiB language model is useful and reasoning performance matters more than broad BF16 fidelity.

It should not be used as the sole source of truth in medical, legal, financial, safety-critical, or other high-stakes settings. Review generated content and evaluate the model on your own workload before deployment.

License and attribution

This derivative is distributed under the Gemma Terms of Use and is subject to the Gemma Prohibited Use Policy. Review the current terms before use or redistribution:

• Gemma 4 license
• Gemma 4 E4B model card
• Upstream Unsloth GGUF repository

Gemma is provided by Google. The upstream BF16 GGUF was published by Unsloth. QLAB produced the mixed-precision allocation; MLAB performed the independent measurement described above.

Citation

@misc{gemma4_2025,
  title     = {Gemma 4},
  author    = {{Gemma Team}},
  year      = {2025},
  publisher = {Google DeepMind},
  url       = {https://ai.google.dev/gemma}
}

Support further QLAB experiments: buymeacoffee.com/hostilevolume