ApacheOne / facelift_convrot_int8_int4_runtimes

FaceLift ConvRot INT8 / INT4 Runtimes

Quantized ConvRot runtimes and transformed checkpoints for FaceLift / OpenFaceLift , tested with native Comfy-Kitchen ConvRot kernels. Four tested configurations are included, ranging from full INT8 to mixed INT4 / INT8 setups.

FP16 reference

Runtime     28.204 s
Peak VRAM    7.828 GiB
Final SSIM   1.0000

Quantized configurations

Official FaceLift examples — turntable videos

All eight official FaceLift examples ( 000 – 007 ) rendered with all four quantization methods.

The four videos in each row use the same input face and the same inference settings.

Example 000

Example 001

Example 002

Example 003

Example 004

Example 005

Example 006

Example 007

---

Official FaceLift examples — multiview outputs

The corresponding six-view MVDiffusion outputs for the same eight official FaceLift examples.

Click any image to open the full-resolution PNG.

Example 000

Example 001

Example 002

Example 003

Example 004

Example 005

Example 006

Example 007

---

Upstream projects

• FaceLift code: https://github.com/weijielyu/FaceLift
• Official OpenFaceLift weights: https://huggingface.co/wlyu/OpenFaceLift
• FaceLift paper: https://arxiv.org/abs/2412.17812
• Comfy-Kitchen: https://github.com/Comfy-Org/comfy-kitchen

---

Benchmark environment

All benchmark results below used the same controlled FaceLift runtime.

GPU             NVIDIA L4
GPU capability  SM 8.9
Python          3.12.13
PyTorch         2.11.0+cu128
Torchvision     0.26.0+cu128
CUDA            12.8
comfy-kitchen   0.2.31
xformers        0.0.35

Benchmark image 004.png
Seed            4
Guidance        3.0
Diffusion steps 50

FaceLift multiview attention used:

XFormersMVAttnProcessor

GSLRM used PyTorch SDPA across all 24 transformer blocks.

The SSIM/PSNR figures below are from the controlled 004.png benchmark. The eight-example gallery above is a broader qualitative comparison across all official FaceLift examples.

---

1. Full INT8 W8A8

Folder:

01_full_int8_w8a8/

Configuration:

CLIP Vision     INT8 W8A8
CLIP Text       INT8 W8A8
UNet            INT8 W8A8
VAE             INT8 W8A8
GSLRM           INT8 W8A8

Quantized Linear layers:

CLIP Vision   193
CLIP Text     138
UNet          218
VAE             8
GSLRM           99
------------------
Total          656

Controlled benchmark:

Runtime          28.044 s
Peak VRAM         5.845 GiB

Multiview:
SSIM              0.997264
PSNR             46.621 dB
MAE               0.4247

Final render:
SSIM              0.992526

This is the highest-quality fully quantized configuration tested.

Official eight-example suite:

Total runtime       256.517 s
Mean / face          32.065 s
Examples completed    8 / 8

---

2. INT4 diffusion + INT8 GSLRM

Folder:

02_int4_diffusion_int8_gslrm/

Configuration:

CLIP Vision     W4A4
CLIP Text       W4A4
UNet            W4A4
VAE             W4A4

GSLRM           INT8 W8A8

Controlled benchmark:

Runtime          25.780 s
Peak VRAM         5.268 GiB

Multiview SSIM    0.902111
Final SSIM        0.903606

Official eight-example suite:

Total runtime       236.716 s
Mean / face          29.589 s
Examples completed    8 / 8

This test showed that the diffusion side can run at W4A4, but the multiview quality loss becomes the dominant bottleneck.

---

3. INT4 diffusion + W4A8 GSLRM body + INT8 heads

Folder:

03_int4_diffusion_w4a8_gslrm_int8_heads/

Configuration:

CLIP Vision                     W4A4
CLIP Text                       W4A4
UNet                            W4A4
VAE                             W4A4

GSLRM body                      W4A8
gaussian_upsampler.linear       INT8 W8A8
pixel_gaussian_decoder.1        INT8 W8A8

GSLRM split:

97 Linear layers   W4A8
 2 Linear layers   INT8 W8A8

Controlled benchmark:

Runtime          26.089 s
Peak VRAM         5.130 GiB

Multiview SSIM    0.902111
Final SSIM        0.902423

Official eight-example suite:

Total runtime       238.067 s
Mean / face          29.758 s
Examples completed    8 / 8

This configuration keeps the GSLRM body at packed 4-bit weights while avoiding A4 activation execution inside GSLRM.

---

4. INT8 diffusion + W4A8 GSLRM body + INT8 heads

Folder:

04_int8_diffusion_w4a8_gslrm_int8_heads/

Configuration:

CLIP Vision                     INT8 W8A8
CLIP Text                       INT8 W8A8
UNet                            INT8 W8A8
VAE                             INT8 W8A8

GSLRM body                      W4A8
gaussian_upsampler.linear       INT8 W8A8
pixel_gaussian_decoder.1        INT8 W8A8

GSLRM split:

97 Linear layers   W4A8
 2 Linear layers   INT8 W8A8

Controlled benchmark:

Runtime          28.139 s
Peak VRAM         5.700 GiB

Multiview:
SSIM              0.997264
PSNR             46.621 dB
MAE               0.4247

Final render:
SSIM              0.984692
PSNR             38.961 dB
MAE               1.2562

Compared directly with full INT8:

Multiview:
MSE               0.0
SSIM              1.0

Final:
SSIM              0.986710
PSNR             40.005 dB
MAE               1.1138

Official eight-example suite:

Total runtime       257.027 s
Mean / face          32.128 s
Examples completed    8 / 8

This is the best mixed INT8 / INT4 configuration tested so far.

---

Main findings

INT8 ConvRot is highly accurate

Full INT8 retained almost all FP16 quality:

FP16 vs INT8 final SSIM = 0.9925

Peak VRAM dropped from:

7.828 GiB -> 5.845 GiB

---

GSLRM is highly sensitive to A4 activations

Running the GSLRM transformer body with A4 activations caused a large reconstruction-quality loss.

Moving GSLRM from:

W4A4 -> W4A8

recovered most of the lost reconstruction quality while retaining packed 4-bit weights.

---

The Gaussian output heads benefit from INT8

The two protected projections are:

gaussian_upsampler.linear
pixel_gaussian_decoder.1

These directly produce the Gaussian reconstruction parameters.

Using INT8 for these layers improves the W4A8 GSLRM configuration.

---

Diffusion becomes the limiting stage at W4A4

With INT4 diffusion:

Multiview SSIM ~= 0.9021

Once information is lost during multiview generation, the downstream GSLRM cannot recover it.

Switching the diffusion stack back to INT8 restores:

Multiview SSIM = 0.9973

and raises final reconstruction quality to:

Final SSIM = 0.9847

---

Recommended configurations

Highest quality

Full INT8 W8A8

Peak VRAM  5.845 GiB
Final SSIM 0.9925

Best mixed quality / memory

INT8 diffusion
+
W4A8 GSLRM body
+
INT8 Gaussian heads

Peak VRAM  5.700 GiB
Final SSIM 0.9847

Lowest-memory mixed configuration tested

INT4 diffusion
+
W4A8 GSLRM body
+
INT8 Gaussian heads

Peak VRAM  5.130 GiB
Final SSIM 0.9024

---

Repository structure

facelift_convrot_int8_int4_runtimes/
│
├── README.md
├── LICENSE
├── build_report.json
│
├── 01_full_int8_w8a8/
│   ├── weights/
│   ├── runtime/
│   ├── outputs/004/
│   ├── examples/official_facelift/
│   │   ├── 000/
│   │   ├── 001/
│   │   ├── ...
│   │   └── 007/
│   ├── manifest.json
│   └── metrics.json
│
├── 02_int4_diffusion_int8_gslrm/
├── 03_int4_diffusion_w4a8_gslrm_int8_heads/
└── 04_int8_diffusion_w4a8_gslrm_int8_heads/

Each official example folder contains:

input.png
multiview.png
output.png
gaussians.ply
turntable.mp4

---

Runtime

Each method contains:

runtime/
├── facelift_convrot_runtime.py
├── run_inference.py
└── requirements.txt

Example:

python runtime/run_inference.py \
    --input_dir /path/to/images \
    --output_dir ./outputs_new

If OpenFaceLift is already downloaded locally:

export OPENFACELIFT_DIR=/path/to/OpenFaceLift

The runtime reconstructs the official model architecture, replaces eligible nn.Linear modules according to manifest.json , loads the transformed ConvRot checkpoints, enables FaceLift's xFormers multiview attention path, and runs the official inference pipeline.

---

ConvRot quantization details

Only eligible rank-2:

nn.Linear

weights are ConvRot quantized.

Convolutions, embeddings, normalization layers, biases, and unsupported tensors remain in their normal precision unless otherwise handled by the upstream model.

ConvRot group selection:

256 preferred
64  fallback
16  fallback

W4 quantization uses:

quant_group_size = 64

W4A4

packed W4 weights
A4 execution
linear_dtype="int4"

W4A8

packed W4 weights
A8 execution
linear_dtype="int8"

INT8 W8A8

INT8 ConvRot weights
INT8 activation execution

---

License

These transformed checkpoints are derived from the official OpenFaceLift model weights.

The official OpenFaceLift weights are distributed under the Adobe Research License .

Review the original license before redistribution or use:

https://huggingface.co/wlyu/OpenFaceLift/blob/main/LICENSE

FaceLift source code is distributed separately under the licensing terms of the official code repository.

This repository is an experimental quantization/runtime derivative and is not the official FaceLift release.

---

References

1. FaceLift — official code
   https://github.com/weijielyu/FaceLift

2. OpenFaceLift — official weights
   https://huggingface.co/wlyu/OpenFaceLift

3. FaceLift paper
   https://arxiv.org/abs/2412.17812

4. Comfy-Kitchen
   https://github.com/Comfy-Org/comfy-kitchen

---

Citation

If you use FaceLift, cite the original FaceLift work:

@InProceedings{FaceLift,
    author    = {Lyu, Weijie and Zhou, Yi and Yang, Ming-Hsuan and Shu, Zhixin},
    title     = {FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2025},
    pages     = {12691-12701}
}