chfm / Qwen-Image-2.1-Text-Encoder-Heretic-NVFP4

Qwen-Image-2.1 Text Encoder (Heretic) — NVFP4

Not affiliated with, or endorsed by, Alibaba / Qwen. This is a community derivative of Qwen/Qwen-Image-2.1 , redistributed under the Qwen Research License — a copy is included in this repo as LICENSE , per §3.a of that agreement.

Non-commercial use only (research or evaluation, §1.i / §2.a). Commercial use requires a separate licence from Qwen ( [email protected] , §2.b).

NVFP4 (4-bit weight) build of Qwen-Image-2.1-Text-Encoder-Heretic , the abliterated text encoder of Qwen/Qwen-Image-2.1 .

File: qwen3vl_8b_nvfp4_heretic.safetensors — 5.87 GB (from 16.33 GB bf16)

Ablation (inherited from the bf16 source)

Produced with Heretic directional ablation ( o_proj + down_proj ), 200 trials / 60 startup trials, knee point of the Pareto front. Measured on mlabonne/harmful_behaviors (refusals) and mlabonne/harmless_alpaca (KL). Independently re-checked on the bf16 source: 0/20 refusals, 4/4 benign questions answered correctly.

Full methodology, Pareto table and reproduction command are in the bf16 repo .

Who this is for

Blackwell GPUs (RTX 50xx, GB10). NVFP4 maps onto native FP4 tensor cores, so you get both the memory saving and a throughput win.

On anything else — including Apple Silicon — ComfyUI falls back to dequantize-then-compute : you keep the memory saving but get no speedup , and it is slightly slower than bf16. On a Mac use the GGUF build instead.

What is quantized, what is protected

Recipe decoded from Comfy-Org's own qwen3vl_8b_w4a8.safetensors and reproduced exactly — the same protection scheme is used for every format in this family :

79.2 % of parameters go to 4-bit, 14.2 % stay at 8-bit, 6.6 % stay at bf16.

Format details

Written in ComfyUI's native quantized-checkpoint layout:

plus a _quantization_metadata entry in the file metadata ( {"format_version": "1.0", "layers": {…}} ).

Produced with ComfyUI's own TensorCoreNVFP4Layout.quantize() — not a reimplementation — so the layout is guaranteed compatible.

Usage (ComfyUI)

Place in ComfyUI/models/text_encoders/ , then:

CLIPLoader → type: qwen_image → TextEncodeQwenImage21

Requires a ComfyUI build with QwenImage21 support (merged after 2026-09-14). Older builds report UNSUPPORTED DIFFUSION MODEL for Qwen-Image-2.1 in general, including the stock bf16 files.

Verified end-to-end: 1024×1024, 25 steps, ~22 s on a GB10 paired with a self-quantized NVFP4 DiT.

Quantization does not fight the ablation

NVFP4 round-trip error measured per layer group:

Directional ablation is a rank-1 edit; it does not create outliers, so the same recipe applies to ablated and stock weights alike.

Pitfalls found while building this

Each of these produces a valid-looking file that is silently wrong — same size, same tensor count, same format strings. They were only caught by diffing against the official release, or by actually running the model.

1. The vision tower must be excluded. Comfy-Org leaves all 351 vision tensors in bf16; a naive "quantize every 2-D weight" pass eats them.
2. int8_tensorwise needs convrot=True explicitly. The 4-bit path applies convolution rotation internally; the INT8 path defaults to False .
3. comfy_quant must serialize the whole per-layer config , not just {"format": …} , or convrot / convrot_groupsize are dropped.
4. MXFP8 scales must be stored as uint8 . TensorCoreMXFP8Layout.quantize() returns float8_e8m0 , which ComfyUI's safetensors loader cannot parse ( KeyError: 'F8_E8M0' ).
5. Comfy-Org's repack strips the model.language_model. prefix. Quantizing straight from the HF layout yields keys ComfyUI never finds — the model loads "successfully" and emits noise.

The rest of this family