AiArtLab / sdxl_vae

SDXL-VAE finetuned

Train status, in progress:

VAE Training Process

• Encoder: Frozen (to avoid retraining SDXL for the new VAE).
• Dataset: 100,000 PNG images
• Training Time: 4 days
• Hardware: Single RTX 4090
• Resolution: 512px
• Precision: FP32
• Effective Batch Size: 16 (batch size 2 + gradient accumulation 8)
• Optimizer: AdamW (8-bit)

Implementation

• Base Code: Used a simple diffusion model training script.
• Training Target: Only the decoder, focusing on image reconstruction.

Loss Functions

• Initially used LPIPS and MSE.
• Noticed FID score improving, but images becoming blurry (FID overfits to blurry images—improving FID is not always good).
• Switched to MAE.
• Balanced LPIPS and MAE at 90/10 ratio.
• Used median perceptual_loss_weight for better balance.

Compare

https://imgsli.com/NDA3Njgw/2/3

Donations

Please contact with us if you may provide some GPU's or money on training

DOGE: DEw2DR8C7BnF8GgcrfTzUjSnGkuMeJhg83

BTC: 3JHv9Hb8kEW8zMAccdgCdZGfrHeMhH1rpN

Contacts

recoilme