AtomicChat / Muse-Glimmer-30B-GGUF

Muse Glimmer 30B — GGUF

GGUF builds of Meta's Muse Glimmer 30B, made from the original BF16 weights.

This is the first stage: full-precision base, 8-bit builds, and measurements. The full quant ladder with a calibrated imatrix and NVFP4 is in progress.

Support was merged into llama.cpp master in #26841 (commit 62bf73d ). You need a build from master — the last tagged release predates it.

Files

Q8_0 fits on a single RTX 5090 and runs at 35.7 t/s. Adding the mmproj for image input pushes you to a second card.

The two experiments/ files exist to answer one question — which of the two big tensors is worth keeping in BF16 — and are measured below. For normal use pick Q8_0 or AD-Q8_0 .

Per-tensor composition of every file is in layouts/*.json . Raw logs behind every number on this page: AtomicChat/Muse-Glimmer-30B-GGUF-metrics .

Running it

Build from master (or grab a nightly binary from the llama.cpp releases page once one includes commit 62bf73d ):

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON
cmake --build build -j --target llama-server llama-mtmd-cli

--jinja is required. Without it neither the chat template nor tool-call parsing works.

llama-server -m Muse-Glimmer-30B-Q8_0.gguf \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on \
    --temp 1.0 --top-p 0.95 --top-k 64

With speculative decoding:

llama-server -m Muse-Glimmer-30B-Q8_0.gguf \
    -md dflash-Muse-Glimmer-30B-BF16.gguf \
    --spec-type draft-dflash --spec-draft-n-max 15 \
    --mmproj mmproj-Muse-Glimmer-30B-BF16.gguf \
    --jinja -ngl 99 -c 16384 -fa on

Reasoning depth is set by a line in the system prompt, not a flag: Reasoning strength: low|medium|high|xhigh. Default is high .

Speed

Hardware: 4x RTX 5090, 2x EPYC 9654, driver 590.48.01.

Without the drafter — llama-bench -p 512,4096 -n 128 -r 3 -ngl 99 -fa 1 :

With the DFlash drafter — llama-server , 9-prompt set (code / prose / reasoning), greedy sampling. Generation speed here is measured end to end by the server, so it is not directly comparable to llama-bench tg128 above:

Acceptance depends heavily on the task: code prompts reach 0.37, free prose drops to 0.12. Meta reports 3.1x on a single 5090; we see 1.9x, most likely because a 4-GPU layer split makes each target forward pass relatively more expensive against drafter overhead.

Quality

Reference is our own BF16 GGUF, which is bit-identical to the original weights (verified by UUID and all 731 tensor hashes). Measured with llama-perplexity --kl-divergence . Lower KLD is better, higher top-1 is better.

Neutral text — 30 languages, 97 windows of 4096, base PPL 5.4614:

Agentic dialogues — native chat markup with tool calls, 85 windows of 4096, base PPL 2.4462:

_{* these two builds live in experiments/ — they isolate one tensor each and are not meant for general use.}

Calibration

imatrix/muse-glimmer.imatrix.gguf is the importance matrix used for the quant ladder. It is published so anyone can rebuild these files, or build their own quants on the same calibration.

How it was collected

An importance matrix records, for every weight matrix in the model, the mean squared activation arriving on each of its input channels. The quantiser uses those numbers to weight reconstruction error when it searches for block scales: error on a channel that usually carries large activations is penalised more than error on a channel that is usually near zero.

Two flags there are off by default and both matter. Without --parse-special , <|start|> and friends are tokenized as literal punctuation, so 40% of the corpus would have been calibrated on sequences the model never produces. Without --process-output , output.weight — 1.34 B parameters sitting directly under the logit softcap — gets no importance data at all.

Corpus and raw logs: AtomicChat/calib-corpora and AtomicChat/Muse-Glimmer-30B-GGUF-metrics .

What the calibration shows

Muse Glimmer has 52 layers with 8 collected matrices each, plus the output matrix, for 417 importance vectors. They reduce to five distinct profiles per layer : attn_q , attn_k , attn_v and attn_gate all read the output of attn_norm , so their importance vectors are identical to the last digit, and ffn_gate and ffn_up likewise share the output of ffn_pre_norm .

For each vector we measured how much of the total importance sits in its strongest channels.

The Y axis lists the five profiles. The X axis is the percentage of a tensor's total importance held by its 10 strongest input channels, and by its strongest 1% of channels, averaged over all 52 layers.

Read against a uniform distribution, the numbers are extreme. Ten channels out of 6656 would hold 0.15% of importance if it were spread evenly; on the attention input they hold 38.8%, a factor of 259. For ffn_down the ten strongest of 19 968 channels hold 19.8% against a uniform 0.05%, a factor of 396. Every group is at least two orders of magnitude away from uniform.

Concentration is not constant with depth.

Here the X axis is the layer index, 0 to 51, and the Y axis is the share of ffn_down importance held by its 10 strongest channels out of 19 968. Layers 5 through 11 sit between 1.6% and 3.6%. Layers 41 and 42 reach 59% and 60% — ten channels out of nearly twenty thousand carrying three fifths of everything, some 1200 times a uniform share. Layers 0 and 4 are isolated spikes in an otherwise flat early region.

Why this matters for quantization

K-quants assign one scale to a block of 32 adjacent weights along the input dimension. A single dominant channel inside a block forces that scale upward and costs precision on the other 31 weights. Concentration therefore makes a block harder to quantize, not easier, and it marks where calibration has the most leverage: the gap between a calibrated and an uncalibrated quant should be widest exactly where the top-10 share is highest.

That is a prediction, not a result. The ladder below tests it by holding everything constant except ffn_down in layers 36 through 47.

One further note on attn_gate . Gated attention is new to this architecture and adds a 6656 x 4096 matrix per layer, roughly a third of the attention parameters. No published quantization work has had importance data for it. Its profile is identical to attn_q , attn_k and attn_v , which means it cannot be distinguished from them by importance alone — any differentiation has to come from measurement, not from the matrix.

Reproducing it

llama-imatrix -m Muse-Glimmer-30B-BF16-00001-of-00002.gguf \
    -f calib_train.txt -o part.gguf \
    -c 8192 -b 2048 -ngl 99 \
    --parse-special --process-output --no-ppl

--no-ppl is required at long context: the perplexity path allocates n_ctx x n_vocab x 4 bytes for logits, which is 26.5 GB at 32768 context with this model's 202 048 token vocabulary. Perplexity is not used by the importance collection, so disabling it costs nothing.

Merging is exact addition of sums and counts, so the corpus can be split across machines and recombined:

llama-imatrix -m any-model.gguf -c 8192 \
    --in-file part-00.gguf,part-01.gguf,part-long.gguf \
    -o muse-glimmer.imatrix.gguf

Per-tensor statistics for the merged file are in imatrix/imatrix-stats.txt in the metrics dataset, produced with llama-imatrix --show-statistics .

What's next

• Full ladder, roughly 20 rungs, with a calibrated imatrix
• NVFP4 for Blackwell
• Code corpus metrics, per-task acceptance breakdown, drafter quantization sweep
• Republished once PR #26841 is merged

Credits

Model by Meta Superintelligence Lab. llama.cpp support by @pcuenca and the ggml-org reviewers.