thedeoxen / refcontrol-FLUX.2-klein-9B-reference-normal-lora

RefControl FLUX.2 Klein 9B – Reference Normal LoRA

📝 Short description

A LoRA for FLUX.2 Klein 9B Base that fuses a reference image (identity) with a normal map (structure / surface orientation).
It preserves identity and style from the reference while following the 3D shape and composition from the normal control map.

Trigger word: refcontrol

📊 Examples

Each preview is a single combined image from ComfyUI: Normal → Reference → Result (left to right).

Normal → Reference → Result

📖 Extended description

This LoRA was primarily trained on humans , but it also works with stylized characters and some objects.
Its main purpose is to preserve identity — facial features, hairstyle, clothing, or object details — from the reference image, while adapting the subject to the structure and surface orientation defined by the normal map.

FLUX.2 Klein 9B Base already handles reference + normal-guided transfer reasonably well with the right prompt alone. This LoRA builds on that capability — it improves consistency and better preserves character identity and normal-map fidelity than the base model without LoRA.

Part of the RefControl family: reference + control fusion for consistent, controllable generation on FLUX.2 Klein 9B Base .

⚙️ How to use

1. Use the first image as the normal map (structure / surface orientation).
2. Use the second image as the reference (character, person, or object).
3. Add the trigger word refcontrol in your prompt.
4. Adjust LoRA weight (recommended 0.8–1.0 ) depending on how strongly you want to preserve identity.

ComfyUI requirements

Normal map extraction in the included workflow uses DSINE-NormalMapPreprocessor (via comfyui_controlnet_aux ):

https://github.com/Fannovel16/comfyui_controlnet_aux

You can disable the built-in normal preprocessor if your input is already a normal map image.

Base model

Trained on and recommended with black-forest-labs/FLUX.2-klein-base-9B .
The undistilled Base variant is intended for LoRA training and custom pipelines (~50 inference steps, guidance_scale ~4.0).

The LoRA also works with the 4-step distilled black-forest-labs/FLUX.2-klein-9B for faster inference (~4 steps, guidance_scale ~1.0), but quality may be slightly lower — especially for identity and normal-map fidelity — compared to the Base model.

✅ Example prompt

refcontrol

🎯 What it does

• Preserves character or object identity across generations.
• Adapts the subject to a new structure or composition defined by the normal map.
• Works best when the normal map has similar proportions and scale to the reference.

⚡ Tips

• Best results when the normal map is not drastically different in body scale or framing from the reference.
• Combine with text prompts to refine background, lighting, or mood.
• Normal maps with clear surface orientation and consistent lighting work well as control input.

📌 Use cases

• Character restyling while keeping identity and normal-defined structure.
• Consistent character design across different compositions.
• Illustration and storyboard generation with normal-guided layout.
• Object transformations with surface-orientation-guided placement.

📦 Files

• Weights: flux2_klein_9b_refcontrol_normal.safetensors
• ComfyUI workflow: refcontrol_normal_flux_klein_9b.json
• Repo: thedeoxen/refcontrol-FLUX.2-klein-9B-reference-normal-lora