satellitedown / fafstmobel

fafstmobel

A locally built 27B derivative of the Swift abliterated Qwen3.8-27B checkpoint: Huihui's observed abliteration delta transplanted onto Swift, quantized to an NVFP4/FP8 text allocation, and exported to a single NInfer v3 artifact with the published DFlash2 draft component bundled alongside vision and MTP.

Built end to end on one workstation (RTX 5090, CUDA 13.1, Python 3.11) by a reproducible pipeline. Every number below was measured on that machine.

Artifact

What was changed relative to upstream

1. Delta transplant onto Swift ( scripts/abliterate.py ). Huihui's abliteration was not re-derived; the base→Huihui weight delta was applied to Swift: result = (swift.float() + (huihui.float() - base.float())).to(bfloat16) . Exactly 70 tensors changed, in language-model layers 17–51 : mlp.down_proj plus self_attn.o_proj (full-attention layers) or linear_attn.out_proj (linear-attention layers). Max |Δ| = 2.83e-2, mean |Δ| = 2.02e-4, 4,220,518,400 elements rewritten. The other 1129 tensors — all vision, MTP, embeddings, norms and the output head — are byte-identical to the Swift checkpoint.
2. Text quantization to the published NVFP4/FP8 allocation: 168 NVFP4 matrices (MLP gate/up/down, group size 16, 4-bit weights, local-dynamic 4-bit activations with calibrated input scales) and 233 FP8 per-channel matrices (attention q/k/v/o, GDN in/out projections, output head, MLP layers 56–63). 32 UltraChat-200k calibration samples × 2048 tokens. Vision, MTP and GDN gate projections stay unquantized.
3. Export to NInfer v3 with the official tools.convert (NInfer revision 98dada0e03cb073fd07f905400b5904bc6e82759 ) and the unmodified qwen3_8_27b_nvfp4 recipe, including the maintained Qwen3.8 chat template.
4. Naming normalization of the text intermediate ( model.language_model.* → model.* ) so the checkpoint is consistent with its flattened text-only config; only safetensors headers were rewritten, verified against per-shard payload digests.

Measured results (RTX 5090, 32 GB; context 16,384; fp8 KV; CUDA graphs on)

Autoregressive throughput varies with machine state across runs; both speculation acceptance rates reproduced exactly in every run.

Usage

ninfer-serve fafstmobel.ninfer --host 127.0.0.1 --port 8088 --model-id fafstmobel \
  --spec dflash2 --draft-tokens 7 --lm-head-draft --vision \
  --kv-dtype fp8 --max-context 16384 --kv-capacity 16384 --max-concurrency 1

ninfer fafstmobel.ninfer --prompt "What is 17 multiplied by 19?" --greedy --max-new 32 \
  --no-thinking --vision --kv-dtype fp8 --max-context 16384 --spec dflash2 --draft-tokens 7

Licensing and notices

This artifact contains three upstream contributions, each retained under its own terms:

• Swift contribution ( ukisai/Swift-Qwen3.8-27b ) — Swift Open License v1.0 , bundled as LICENSE.swift . That licence grants reproduction and distribution of derivative works, including converted weights, subject to its conditions: recipients receive a copy of the licence, modified files carry prominent change notices, attribution notices are retained, and Commercial Use is not licensed for a Legal Entity exceeding the one-million-USD Threshold (Section 5) . A separate Swift Enterprise License is required above that threshold. This derivative is distributed under the same terms for the Swift contribution.
• Base model ( Qwen/Qwen3.8-27B ) — Apache-2.0 , bundled as LICENSE-APACHE-2.0 . Qwen3.8-27B portions remain under the Apache License 2.0.
• Huihui abliteration delta ( huihui-ai/Huihui-Qwen3.8-27B-abliterated ) — Apache-2.0, bundled as LICENSE.huihui . Only the observed weight difference was transplanted.
• DFlash2 draft ( z-lab/Qwen3.8-27B-DFlash2 ) — Apache-2.0; bundled by conversion, unmodified and unquantized in its file.
• NInfer ( https://github.com/Neroued/ninfer , Apache-2.0) — the converter and runtime used.

NOTICE carries the upstream notice plus this derivative's transformation notice; PROVENANCE.md records the exact repository revisions.

Claim boundaries

• This is a delta transplant , not a newly fitted refusal direction, and it is not proof that all refusals are removed. Behavioural claims about refusal removal are not made here.
• The DFlash2 draft is pretrained upstream and bundled by conversion; no drafter was trained, and Swift was not retrained.
• The quantization is a reproduction of the published allocation, not a bit-exact replication of any other derivative; no other derivative's weights were downloaded or compared.
• Not suitable for safety-critical use. Outputs are unmoderated model text.