exploitintel / cve-cwe-gemma4-12b-GGUF

cve-cwe-gemma4-12b-GGUF

GGUF quantizations of exploitintel/cve-cwe-gemma4-12b — a Gemma 4 12B fine-tune that maps a CVE description to its CWE ID(s) (MITRE View-1003).

📖 Write-up: From Essays to CWE-319 — how this fine-tune beats stock Gemma 4 at CWE classification

• Input: a free-text vulnerability description.
• Output: comma-separated CWE IDs, e.g. CWE-79 or CWE-89, CWE-352 .
• Text-only build (vision/audio projectors omitted).

Files

Requires a current llama.cpp / Ollama build with Gemma 4 ( gemma4_unified ) support.

Quality vs the full model

Held-out test split (10,514 examples), description-only , greedy. The bf16 column is the unquantized transformers model; Q8/Q4 are measured through Ollama.

Choosing a quant: Q8_0 is effectively lossless. Q4_K_M costs only 1.5 points on exact/micro but ** 7 points on macro-F1** — i.e. it degrades the rare-CWE long tail specifically. Use Q8_0 if rare/long-tail CWEs matter; use Q4_K_M if size/speed dominate and you mostly see common CWEs.

Usage — Ollama

# build the model from the GGUF (see the Modelfile below)
ollama create cve-cwe-gemma4 -f Modelfile

# IMPORTANT: gemma4 runs with "thinking" ON by default, which wastes ~100-900
# tokens per query for this single-label task. Disable it:
ollama run cve-cwe-gemma4
>>> /set nothink
>>> The update handler transmits user credentials over an unencrypted HTTP channel.
CWE-319

Via the API, pass "think": false (the response field is the clean answer):

curl -s http://localhost:11434/api/generate -d '{
  "model": "cve-cwe-gemma4",
  "prompt": "SQL injection via the username parameter in the login form.",
  "stream": false,
  "think": false
}'
# -> {"response":"CWE-89", ...}

Modelfile

FROM ./cve-cwe-gemma4-12b-Q8_0.gguf

TEMPLATE """{{ if .System }}<|turn>system
{{ .System }}<turn|>
{{ end }}<|turn>user
{{ .Prompt }}<turn|>
<|turn>model
<|channel>thought
<channel|>{{ .Response }}<turn|>
"""

SYSTEM """You are a vulnerability analyst. Given a CVE description, reply with only the CWE ID(s) it maps to, comma-separated."""

PARAMETER temperature 0
PARAMETER stop "<turn|>"
PARAMETER stop "<|turn>"

Usage — llama.cpp

llama-cli -m cve-cwe-gemma4-12b-Q8_0.gguf --jinja \
  -sys "You are a vulnerability analyst. Given a CVE description, reply with only the CWE ID(s) it maps to, comma-separated." \
  -p "A heap-based buffer overflow occurs when parsing a crafted font file."

Limitations

Description-only triage aid; outputs are suggestions, not authoritative classifications. Keep a human in the loop for security-relevant decisions. Scope is MITRE View-1003. See the base model card for training details.

License

Apache-2.0, inherited from the Gemma 4 base model.