wangkanai / wan22-fp16

WAN 2.2 FP16 - Image-to-Video Models

High-quality image-to-video (I2V) generation models in full FP16 precision with GGUF quantized variants, featuring advanced camera control and enhancement LoRAs for maximum quality video generation.

Model Description

WAN 2.2 FP16 is a 14-billion parameter video generation model based on diffusion architecture, providing full FP16 precision for maximum quality image-to-video generation. This repository contains FP16 models for research and archival quality, plus GGUF quantized variants for memory-constrained deployments.

Key Features :

• 14B parameter diffusion-based architecture
• Full FP16 precision for maximum quality (27GB per model)
• GGUF quantized variants (Q4, Q8) for memory efficiency
• Dedicated VAE for video latent encoding/decoding
• Extensive LoRA ecosystem for camera control (v2) and visual enhancement
• Support for both high-noise (creative) and low-noise (faithful) generation modes
• Image-to-video capabilities with cinematic camera control

Model Statistics :

• Total Repository Size : ~88GB
• Model Architecture : Diffusion-based image-to-video generation
• Supported Formats : .safetensors (FP16), .gguf (Q4/Q8 quantized)
• Parameters : 14 billion
• Precision : FP16 (full precision) + GGUF Q4/Q8 quantization
• Input : Images + text prompts
• Output : Video sequences (typically 16-24 frames)

How to Get Started with the Model

Quick start example for image-to-video generation with FP16:

from diffusers import DiffusionPipeline, AutoencoderKL
import torch
from PIL import Image

# Load your input image
input_image = Image.open("your_image.jpg")

# Load pipeline with FP16 precision
pipe = DiffusionPipeline.from_pretrained(
    "base-model-path",
    torch_dtype=torch.float16
)

# Load WAN 2.2 FP16 VAE
pipe.vae = AutoencoderKL.from_single_file(
    "E:/huggingface/wan22-fp16/vae/wan/wan22-vae.safetensors"
)

# Load I2V FP16 model (maximum quality)
pipe.unet = torch.load(
    "E:/huggingface/wan22-fp16/diffusion_models/wan/wan22-i2v-high-noise-14b-fp16.safetensors"
)

pipe.to("cuda")

# Generate video
video = pipe(
    image=input_image,
    prompt="cinematic shot, high quality",
    num_inference_steps=50,
    num_frames=16
).frames

# Save video
from diffusers.utils import export_to_video
export_to_video(video, "output.mp4", fps=8)

For detailed usage examples including camera control, GGUF models, and LoRA combinations, see the Usage section below.

Directory Structure

wan22-fp16/
├── diffusion_models/wan/    # FP16 and GGUF I2V models
├── loras/wan/               # Camera control (v2), action, and enhancement LoRAs
└── vae/wan/                 # Video VAE for latent encoding/decoding

Models

Base Diffusion Models

Located in diffusion_models/wan/

Image-to-Video (I2V) Models

FP16 Precision (Maximum Quality):

GGUF Quantized (Memory Efficient):

Note : This repository contains I2V (image-to-video) models only. For T2V (text-to-video) models, see wan22-fp8 repository.

Video VAE

Located in vae/wan/

• File : wan22-vae.safetensors
• Size : 1.4GB
• Purpose : Video latent encoder/decoder for compressing video frames

Enhancement LoRAs

Located in loras/wan/

Camera Control LoRAs (v2 - Enhanced)

Visual Enhancement LoRAs

Action LoRAs

Usage

Basic Image-to-Video Generation (FP16)

from diffusers import DiffusionPipeline, AutoencoderKL
import torch
from PIL import Image

# Load input image
input_image = Image.open("path/to/your/image.jpg")

# Load I2V pipeline with FP16 precision
pipe = DiffusionPipeline.from_pretrained(
    "path-to-base-model",
    torch_dtype=torch.float16
)

# Load WAN 2.2 FP16 I2V model (maximum quality)
pipe.unet = torch.load(
    "E:/huggingface/wan22-fp16/diffusion_models/wan/wan22-i2v-high-noise-14b-fp16.safetensors"
)

# Load WAN 2.2 VAE
pipe.vae = AutoencoderKL.from_single_file(
    "E:/huggingface/wan22-fp16/vae/wan/wan22-vae.safetensors"
)

pipe.to("cuda")

# Generate video from image
video = pipe(
    image=input_image,
    prompt="cinematic shot, high quality, detailed",
    num_inference_steps=50,
    num_frames=16
).frames

# Save video
from diffusers.utils import export_to_video
export_to_video(video, "output.mp4", fps=8)

Using Camera Control LoRAs

# After loading base pipeline, add camera control
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp16/loras/wan/wan22-camera-rotation-rank16-v2.safetensors"
)

# Generate with camera movement
video = pipe(
    image=input_image,
    prompt="rotating camera around a sculpture, cinematic",
    num_inference_steps=50
).frames

Combining Multiple LoRAs

# Load multiple LoRAs with different weights
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp16/loras/wan/wan22-camera-drone-rank16-v2.safetensors",
    adapter_name="camera_drone"
)
pipe.load_lora_weights(
    "E:/huggingface/wan22-fp16/loras/wan/wan22-light-volumetric.safetensors",
    adapter_name="volumetric_light"
)

# Set LoRA weights
pipe.set_adapters(["camera_drone", "volumetric_light"], adapter_weights=[0.8, 0.6])

# Generate with combined effects
video = pipe(
    image=input_image,
    prompt="aerial drone shot with volumetric lighting at sunset",
    num_inference_steps=50
).frames

Using GGUF Quantized Models

# For 12GB VRAM GPUs, use GGUF Q4 models
# Note: GGUF models may require specific loaders
# Check llama.cpp or gguf-specific loading documentation

# Example with GGUF model (loader-dependent)
pipe.unet = load_gguf_model(
    "E:/huggingface/wan22-fp16/diffusion_models/wan/wan22-i2v-a14b-highnoise-q4-k-s.gguf"
)

Model Selection Guide

Precision Trade-offs (This Repository)

FP16 Models (Available in this repo):

• ✅ Best quality and detail (maximum precision)
• ✅ No quantization artifacts
• ✅ Research-grade quality
• ❌ Largest file size (27GB per model)
• ❌ Requires 24GB+ VRAM (RTX 4090, A5000, A6000)
• 🎯 Use for: Final production, research, archival quality, maximum fidelity

GGUF Q8 (Available in this repo):

• ✅ Medium size (15GB)
• ✅ Better quality than Q4
• ✅ Works on 16GB VRAM GPUs
• ❌ Some quality degradation vs FP16
• 🎯 Use for: Balance between quality and memory efficiency

GGUF Q4_K_S (Available in this repo):

• ✅ Smallest size (8.2GB)
• ✅ Works on 12GB VRAM GPUs
• ✅ Fastest inference
• ❌ More quality degradation than Q8 or FP16
• 🎯 Use for: Memory-constrained systems (12GB VRAM), rapid prototyping, testing

FP8 Models (Not in this repo):

• See wan22-fp8 repository for FP8 variants
• 14GB per model, requires 16GB+ VRAM
• Excellent quality/VRAM balance for production deployment

Noise Schedule Selection

High-Noise Models :

• More creative interpretation
• Better for abstract or stylized content
• Higher variance in outputs

Low-Noise Models :

• More faithful to input/prompt
• Better for realistic content
• More consistent results

Hardware Requirements

System Requirements :

• CUDA 11.8+ or 12.1+
• PyTorch 2.0+ (FP16 support standard)
• diffusers library 0.20+
• 88GB free disk space (full repository)
• 32GB+ system RAM recommended

Note : This repository contains I2V models only. For T2V or FP8 models, see wan22-fp8 repository.

Performance Tips

1. Memory Optimization :

   • Use GGUF Q4 models on 12GB GPUs for maximum efficiency
   • Use GGUF Q8 for balance between quality and VRAM (16GB+ GPUs)
   • FP16 models require 24GB+ VRAM but provide best quality
   • Enable gradient checkpointing if fine-tuning
   • Use CPU offloading if needed: pipe.enable_model_cpu_offload()

2. Quality Optimization :

   • FP16 provides maximum quality in this repository
   • Combine multiple LoRAs at weights 0.6-0.8 for best results
   • Experiment with both high and low noise variants
   • For production quality, prioritize FP16 models when VRAM allows
   • GGUF Q8 provides good quality for most use cases

3. Speed Optimization :

   • GGUF Q4 models fastest for rapid prototyping
   • Reduce num_inference_steps to 20-30 for testing
   • Enable xformers attention: pipe.enable_xformers_memory_efficient_attention()
   • FP16 models benefit from tensor core acceleration on modern GPUs

4. GPU-Specific Tips :

   • RTX 4090/A6000 : Use FP16 for maximum quality
   • RTX 4080/3090 : Choose between GGUF Q8 (16GB) based on quality needs
   • RTX 4070 Ti/3080 : Use GGUF Q4 exclusively
   • Research/Archival : Always use FP16 models

Prompting Guidelines

Camera Movement Prompts

Rotation : "rotating camera", "camera circles around", "360-degree view", "orbital camera"

Arc Shot : "arc shot", "curved camera movement", "sweeping motion", "cinematic arc"

Drone : "aerial view", "drone shot", "bird's eye view", "flying camera", "overhead shot"

Zoom : "zooming out", "zoom in on subject", "dolly zoom"

Enhancement Prompts

Volumetric Lighting : "volumetric light rays", "god rays", "atmospheric lighting", "light shafts"

Face Natural : Use with portrait videos for more realistic facial expressions and movements

File Formats

• .safetensors : Secure tensor format, recommended for most use cases
• .gguf : Quantized format for memory-constrained environments

Intended Uses

Direct Use

WAN 2.2 is designed for:

• Content Creation : Generate videos from text descriptions or images for creative projects, advertising, and entertainment
• Prototyping : Rapid video concept visualization for storyboarding and pre-production
• Research : Academic research in video generation, diffusion models, and controllable video synthesis
• Application Development : Building video generation features in applications and services

Downstream Use

• Fine-tuning on domain-specific video datasets
• Integration with video editing pipelines
• Custom LoRA development for specialized camera movements or visual effects
• Video dataset augmentation and synthetic data generation

Out-of-Scope Use

The model should NOT be used for:

• Generating deceptive, harmful, or misleading video content
• Creating deepfakes or non-consensual content of individuals
• Producing content that violates copyright or intellectual property rights
• Generating content intended to harass, abuse, or discriminate
• Creating videos for illegal purposes or activities

Bias, Risks, and Limitations

Known Limitations

Technical Limitations :

• Temporal Consistency : May produce flickering or inconsistent motion in long sequences
• Fine Details : Small objects or intricate textures may lack detail or consistency
• Physical Realism : Generated physics may not always follow real-world rules (gravity, momentum, etc.)
• Text Rendering : Cannot reliably render readable text within generated videos
• Face Quality : Faces may show artifacts or unnatural movements (mitigated by face-naturalizer LoRA)
• Memory Requirements : High VRAM requirements limit accessibility (12-32GB depending on precision)

Content Limitations :

• Training data biases may affect representation of diverse demographics, cultures, and scenarios
• May struggle with uncommon objects, rare scenarios, or niche content
• Camera control may not always precisely match intended movements
• Generated content may reflect biases present in training data

Risks and Mitigations

Misuse Risks :

• Deepfakes and Misinformation : Model could be used to create deceptive content
  • Mitigation : Implement content authentication, watermarking, and usage monitoring
• Copyright Infringement : May generate content similar to copyrighted material
  • Mitigation : Avoid training on copyrighted data, implement content filtering
• Harmful Content : Could generate disturbing or inappropriate content
  • Mitigation : Implement safety filters, content moderation, and responsible use guidelines

Ethical Considerations :

• Users should obtain appropriate permissions before generating videos of identifiable individuals
• Generated content should be clearly labeled as AI-generated to prevent deception
• Consider environmental impact of compute-intensive inference
• Respect privacy, consent, and intellectual property rights

Recommendations

• Implement content moderation and safety filters in production deployments
• Add visible/invisible watermarks to identify AI-generated content
• Provide clear disclaimers that content is AI-generated
• Monitor for misuse and implement usage policies
• Consider accessibility trade-offs when selecting model precision
• Validate outputs for unintended biases or harmful content before distribution

Training Details

Training Data

Training data details are not publicly available. Typical video diffusion models are trained on:

• Large-scale video datasets with diverse content
• Text-video pairs for caption conditioning
• Image-video pairs for image-to-video tasks

Note : Specific training dataset information should be obtained from the original model authors.

Training Procedure

Training Hyperparameters (typical for models of this scale):

• Architecture: Diffusion transformer with 14B parameters
• Precision formats: FP16, FP8, GGUF quantization
• Video VAE: Separate encoder/decoder for latent compression
• LoRA adapters: Rank-16 to rank-64 for camera control

Noise Schedules :

• High-noise models : Greater noise variance for creative generation
• Low-noise models : Lower noise variance for faithful reproduction

Compute Infrastructure

Inference Requirements (This Repository) :

• FP16 : 24-32GB VRAM (NVIDIA A6000, RTX 4090, RTX 6000 Ada, A5000)
• GGUF Q8 : 16-20GB VRAM (NVIDIA RTX 4080, RTX 3090, RTX 4070 Ti Super)
• GGUF Q4 : 12-16GB VRAM (NVIDIA RTX 4070 Ti, RTX 3080, RTX 4060 Ti 16GB)

Environmental Impact

Video generation models require significant computational resources. This FP16/GGUF repository provides options for different quality/efficiency needs:

• Model Size : 88GB total (FP16 + GGUF variants + LoRAs)
• Inference Power :
  • FP16: 350-450W (high-end GPUs)
  • GGUF Q8: 250-350W (mid-range GPUs)
  • GGUF Q4: 150-250W (efficient GPUs)
• Carbon Footprint : Varies by energy source and usage patterns
• Quality Options : FP16 for maximum quality, GGUF for efficiency

Recommendations for Reducing Impact :

• Use GGUF Q4 quantized models for maximum efficiency (8.2GB vs 27GB FP16)
• GGUF Q8 provides good quality/efficiency balance
• Reserve FP16 for final renders and archival quality needs
• Batch process multiple requests to amortize overhead
• Use energy-efficient hardware when possible
• Consider carbon offset for production deployments

License

Please check the original WAN 2.2 model repository for specific license terms and usage restrictions. This repository uses the "other" license tag pending clarification of the original license.

Citation

If you use WAN 2.2 in your research or applications, please cite the original model repository.

BibTeX (template):

@misc{wan22,
  title={WAN 2.2: Image-to-Video and Text-to-Video Generation},
  author={[Original Authors]},
  year={2024},
  howpublished={\url{https://huggingface.co/[original-repo]}},
}

Model Card Authors

This model card was created by the repository maintainer based on available model information and standard Hugging Face model card guidelines.

Model Card Contact

For questions about this model card or repository, please open an issue in the repository or contact the original model authors.

Troubleshooting

Out of Memory Errors :

• Switch from FP16 to GGUF Q8 or Q4 quantized models
• FP16 requires 24GB+ VRAM - if you have less, use GGUF variants
• Reduce num_frames (try 8 or 12 instead of 16)
• Reduce batch size to 1
• Enable CPU offloading: pipe.enable_model_cpu_offload()
• Enable sequential CPU offload: pipe.enable_sequential_cpu_offload()

Quality Issues :

• Try both high-noise and low-noise variants
• If using GGUF Q4, try Q8 or FP16 for better quality (requires more VRAM)
• If using GGUF Q8, try FP16 for maximum quality (requires 24GB+ VRAM)
• Adjust LoRA weights (0.5-1.0 range)
• Increase num_inference_steps (50-100)
• FP16 models provide best quality but require high-end GPUs

Slow Generation :

• GGUF Q4 models are fastest for rapid iteration
• Enable xformers: pipe.enable_xformers_memory_efficient_attention()
• Reduce inference steps to 20-30 for testing
• FP16 models are slower but provide maximum quality
• Consider trading quality for speed by using GGUF variants

GGUF Model Loading Issues :

• Ensure you're using a GGUF-compatible loader
• GGUF models may require specific diffusers versions
• Check llama.cpp or gguf-specific loading documentation
• Some GGUF loaders may not be compatible with diffusers

Support

For issues, questions, or contributions, please refer to the main Hugging Face model repository.

Related Repositories

• wan22-fp8 : FP8 and GGUF quantized I2V + T2V models (88GB, includes text-to-video)
• wan21-fp16 : WAN 2.1 FP16 models (camera control v1, I2V only, 480p/720p)
• wan21-fp8 : WAN 2.1 FP8 models (camera control v1, I2V only, 480p/720p)

Summary

This repository contains WAN 2.2 models in full FP16 precision with GGUF quantized variants for maximum quality image-to-video generation:

• 88GB total (FP16 + GGUF variants + LoRAs)
• FP16 models : 27GB each, maximum quality, requires 24GB+ VRAM
• GGUF Q8 models : 15GB, good quality balance, requires 16GB+ VRAM
• GGUF Q4 models : 8.2GB each, maximum memory efficiency, requires 12GB+ VRAM
• Camera Control v2 : Enhanced camera LoRAs with improved quality
• 9 Enhancement LoRAs : Camera control, lighting, faces, actions (293-586MB each)
• I2V Only : Image-to-video models (for T2V, see wan22-fp8)

Recommended for : Research, archival quality, final production renders, maximum fidelity needs, high-end GPU owners (24GB+ VRAM)

Quality Hierarchy : FP16 > GGUF Q8 > GGUF Q4

---

Last Updated : October 2024 Model Version : WAN 2.2 FP16 Repository Type : Full Precision Model Weights + GGUF Quantized Variants Repository Size : ~88GB Primary Use Case : Maximum quality I2V generation