Rootkit7 / Ternary-Bonsai-4B-abliterated

Ternary-Bonsai-4B-abliterated

An abliterated variant of prism-ml/Ternary-Bonsai-4B-unpacked : the refusal direction has been orthogonalized out of the residual-writing weights, so it no longer represents refusal. Produced for refusal-mechanism, security, and CTF research with the Solutus abliteration toolkit.

⚠️ Safety-reduced model. Refusal has been deliberately removed. It complies with requests a normal instruct model declines. Intended for security research, red-teaming, CTF, and interpretability — not for producing harm. You are responsible for how you use it.

Model details

This is abliteration, not fine-tuning

No gradient training. Abliteration is a weight edit : extract the harmful-vs-harmless activation direction, then project it out of every residual-writing matrix ( W' = (I − rrᵀ)W ). Knowledge is untouched; only the refusal write is removed.

Results (n=40 held-out, batch_size=1)

Abliteration targets layer 28 with 4 SVD directions (the causal refusal layer; an earlier build used layer 31 and left ~2.5% standard + noticeable security refusal — this supersedes it). Verified by an independent reload from disk. Rates only — no harmful completions are distributed.

Capability is intact (0% degenerate output, coherent on general prompts) — a benign example:

Q: Write a haiku about debugging code.
A: A single line fails—
   Logic bends, error glows—
   Silent code screams.

⚠️ Which quant to use (measured — quantization can restore refusal)

The GGUFs were refusal-tested , not just checked for coherence. Aggressive quantization brings back a little refusal on the hardest security prompts:

For security/CTF, use Q8_0, F16, or the safetensors/MLX path for a fully clean 0.000. Q4_K_M is fine for general use but restores ~8% refusal on the hardest exploit prompts. These are ordinary llama.cpp quants — not Bonsai's ternary Q2_0 (that needs Prism's fork, and re-quantizing the abliterated weights onto the ternary grid would distort the edit).

Usage

Transformers (safetensors):

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
m = "Rootkit7/Ternary-Bonsai-4B-abliterated"
tok = AutoTokenizer.from_pretrained(m)
model = AutoModelForCausalLM.from_pretrained(m, dtype=torch.float16, device_map="auto")
msgs = [{"role": "user", "content": "Explain how a buffer overflow works."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(model.device)
print(tok.decode(model.generate(ids, max_new_tokens=400)[0][ids.shape[1]:], skip_special_tokens=True))

llama.cpp / Ollama / LM Studio (GGUF — prefer Q8_0):

llama-cli -m Ternary-Bonsai-4B-abliterated-Q8_0.gguf --jinja -p "Explain how a buffer overflow works."
# Ollama:  FROM ./Ternary-Bonsai-4B-abliterated-Q8_0.gguf  in a Modelfile, then: ollama run ...

Apple Silicon (MLX — runs the safetensors directly, no GGUF needed):

pip install mlx-lm
mlx_lm.generate --model Rootkit7/Ternary-Bonsai-4B-abliterated --prompt "..." --max-tokens 400

Needs a recent transformers (Qwen3; base recorded 4.57.6). This is a Qwen3 thinking model — it emits a <think>…</think> block before the answer.

Intended use & limitations

• Intended: security research, CTF, red-teaming, refusal-mechanism interpretability.
• Not intended: producing harmful content, or deployment where a safety layer is expected.
• Limitations: 4 B model — factual reliability is limited; Q4_K_M restores ~8% refusal on the hardest security prompts (use Q8_0+); abliteration removes behavioral refusal, but the refusal feature is still linearly decodable in activations (not "unlearned").

License & attribution

Apache-2.0, inherited from the base. Created using Bonsai by Prism ML. Base built from Qwen3-4B (© 2024 Alibaba Cloud, Apache-2.0). LICENSE + NOTICE.txt included. Abliteration by Solutus .