wangkanai / wan-2.2

WAN 2.2 - Image-to-Video and Text-to-Video Models

High-quality image-to-video (I2V) and text-to-video (T2V) generation models with advanced camera control and enhancement LoRAs.

Model Description

WAN 2.2 is a 14-billion parameter video generation model based on diffusion architecture, available in multiple precision formats (FP16, FP8, GGUF) optimized for different hardware configurations. The model supports both text-to-video and image-to-video generation with controllable camera movements and visual enhancements through specialized LoRA adapters.

Key Features :

• 14B parameter diffusion-based architecture
• Multiple precision formats for different hardware constraints
• Dedicated VAE for video latent encoding/decoding
• Extensive LoRA ecosystem for camera control and visual enhancement
• Support for both high-noise (creative) and low-noise (faithful) generation modes

Model Statistics :

• Total Repository Size : ~142GB
• Model Architecture : Diffusion-based video generation
• Supported Formats : .safetensors , .gguf
• Parameters : 14 billion
• Input : Text prompts and/or images
• Output : Video sequences (typically 16-24 frames)

How to Get Started with the Model

Quick start example for image-to-video generation:

from diffusers import DiffusionPipeline, AutoencoderKL
import torch
from PIL import Image

# Load your input image
input_image = Image.open("your_image.jpg")

# Load pipeline
pipe = DiffusionPipeline.from_pretrained(
    "base-model-path",
    torch_dtype=torch.float16
)

# Load WAN 2.2 VAE
pipe.vae = AutoencoderKL.from_single_file(
    "./vae/wan/wan22-vae.safetensors"
)

# Load I2V model (FP8 for balanced performance)
pipe.unet = torch.load(
    "./diffusion_models/wan/wan22-i2v-high-noise-14b-fp8-scaled.safetensors"
)

pipe.to("cuda")

# Generate video
video = pipe(
    image=input_image,
    prompt="cinematic shot, high quality",
    num_inference_steps=50,
    num_frames=16
).frames

For detailed usage examples including camera control and LoRA combinations, see the Usage section below.

Directory Structure

wan-2.2/
├── diffusion_models/wan/    # Base I2V and T2V models
├── loras/wan/               # Camera control, action, and enhancement LoRAs
└── vae/wan/                 # Video VAE for latent encoding/decoding

Models

Base Diffusion Models

Located in diffusion_models/wan/

Text-to-Video (T2V) Models

Image-to-Video (I2V) Models

FP16 Precision (Best Quality):

• wan22-i2v-high-noise-14b-fp16.safetensors - 27GB, requires 24GB+ VRAM
• wan22-i2v-low-noise-14b-fp16.safetensors - 27GB, requires 24GB+ VRAM

FP8 Precision (Balanced):

• wan22-i2v-high-noise-14b-fp8-scaled.safetensors - 14GB, requires 16GB+ VRAM
• wan22-i2v-low-noise-14b-fp8-scaled.safetensors - 14GB, requires 16GB+ VRAM

GGUF Quantized (Memory Efficient):

• wan22-i2v-a14b-highnoise-q4-k-s.gguf - 8.2GB, requires 12GB+ VRAM
• wan22-i2v-a14b-lownoise-q4-k-s.gguf - 8.2GB, requires 12GB+ VRAM
• wan22-i2v-a14b-gguf-a14b-high.gguf - 15GB, requires 16GB+ VRAM (higher precision)

Video VAE

Located in vae/wan/

• wan22-vae.safetensors - Video latent encoder/decoder for compressing video frames

Enhancement LoRAs

Located in loras/wan/

Camera Control LoRAs

Visual Enhancement LoRAs

Action LoRAs

Usage

Basic Image-to-Video Generation

from diffusers import DiffusionPipeline, AutoencoderKL
import torch

# Load I2V pipeline with FP8 model (balanced performance)
pipe = DiffusionPipeline.from_pretrained(
    "path-to-base-model",
    torch_dtype=torch.float16
)

# Load WAN 2.2 I2V model
pipe.unet = torch.load(
    "E:/huggingface/wan-2.2/diffusion_models/wan/wan22-i2v-high-noise-14b-fp8-scaled.safetensors"
)

# Load WAN 2.2 VAE
pipe.vae = AutoencoderKL.from_single_file(
    "E:/huggingface/wan-2.2/vae/wan/wan22-vae.safetensors"
)

pipe.to("cuda")

# Generate video from image
video = pipe(
    image=input_image,
    prompt="your prompt here",
    num_inference_steps=50,
    num_frames=16
).frames

Text-to-Video Generation

from diffusers import DiffusionPipeline
import torch

# Load T2V pipeline
pipe = DiffusionPipeline.from_pretrained(
    "path-to-base-model",
    torch_dtype=torch.float16
)

# Load T2V model
pipe.unet = torch.load(
    "E:/huggingface/wan-2.2/diffusion_models/wan/wan22-t2v-low-noise-14b-fp8-scaled.safetensors"
)

pipe.to("cuda")

# Generate video from text
video = pipe(
    prompt="a cat walking through a garden",
    num_inference_steps=50,
    num_frames=16
).frames

Using Camera Control LoRAs

# After loading base pipeline, add camera control
pipe.load_lora_weights(
    "E:/huggingface/wan-2.2/loras/wan/wan22-camera-rotation-rank16-v2.safetensors"
)

# Generate with camera movement
video = pipe(
    image=input_image,
    prompt="rotating camera around a sculpture",
    num_inference_steps=50
).frames

Combining Multiple LoRAs

# Load multiple LoRAs with different weights
pipe.load_lora_weights(
    "E:/huggingface/wan-2.2/loras/wan/wan22-camera-drone-rank16-v2.safetensors",
    adapter_name="camera_drone"
)
pipe.load_lora_weights(
    "E:/huggingface/wan-2.2/loras/wan/wan22-light-volumetric.safetensors",
    adapter_name="volumetric_light"
)

# Set LoRA weights
pipe.set_adapters(["camera_drone", "volumetric_light"], adapter_weights=[0.8, 0.6])

# Generate with combined effects
video = pipe(
    prompt="aerial drone shot with volumetric lighting at sunset",
    num_inference_steps=50
).frames

Model Selection Guide

Precision Trade-offs

FP16 Models :

• ✅ Best quality and detail
• ❌ Largest file size (27GB)
• ❌ Requires 24GB+ VRAM
• 🎯 Use for: Final production, maximum quality

FP8 Models :

• ✅ 50% smaller than FP16 (14GB)
• ✅ Minimal quality loss
• ✅ Faster inference
• ❌ Requires 16GB+ VRAM
• 🎯 Use for: Balanced quality/performance, most users

GGUF Quantized :

• ✅ Smallest size (8-15GB)
• ✅ Works on 12GB VRAM GPUs
• ❌ Some quality degradation
• 🎯 Use for: Memory-constrained systems, testing

Noise Schedule Selection

High-Noise Models :

• More creative interpretation
• Better for abstract or stylized content
• Higher variance in outputs

Low-Noise Models :

• More faithful to input/prompt
• Better for realistic content
• More consistent results

Hardware Requirements

System Requirements :

• CUDA 11.8+ or 12.1+
• PyTorch 2.0+
• diffusers library
• 50GB+ free disk space for FP8 models
• 100GB+ free disk space for FP16 models

Performance Tips

1. Memory Optimization :

   • Use FP8 models for most use cases
   • Enable torch.cuda.amp for mixed precision
   • Use gradient checkpointing if fine-tuning

2. Quality Optimization :

   • Use FP16 for final renders
   • Combine multiple LoRAs at weights 0.6-0.8
   • Experiment with both high and low noise variants

3. Speed Optimization :

   • Use GGUF quantized models for rapid prototyping
   • Reduce num_inference_steps to 20-30 for testing
   • Enable xformers attention optimization

Prompting Guidelines

Camera Movement Prompts

Rotation : "rotating camera", "camera circles around", "360-degree view", "orbital camera"

Arc Shot : "arc shot", "curved camera movement", "sweeping motion", "cinematic arc"

Drone : "aerial view", "drone shot", "bird's eye view", "flying camera", "overhead shot"

Zoom : "zooming out", "zoom in on subject", "dolly zoom"

Enhancement Prompts

Volumetric Lighting : "volumetric light rays", "god rays", "atmospheric lighting", "light shafts"

Face Natural : Use with portrait videos for more realistic facial expressions and movements

File Formats

• .safetensors : Secure tensor format, recommended for most use cases
• .gguf : Quantized format for memory-constrained environments

Intended Uses

Direct Use

WAN 2.2 is designed for:

• Content Creation : Generate videos from text descriptions or images for creative projects, advertising, and entertainment
• Prototyping : Rapid video concept visualization for storyboarding and pre-production
• Research : Academic research in video generation, diffusion models, and controllable video synthesis
• Application Development : Building video generation features in applications and services

Downstream Use

• Fine-tuning on domain-specific video datasets
• Integration with video editing pipelines
• Custom LoRA development for specialized camera movements or visual effects
• Video dataset augmentation and synthetic data generation

Out-of-Scope Use

The model should NOT be used for:

• Generating deceptive, harmful, or misleading video content
• Creating deepfakes or non-consensual content of individuals
• Producing content that violates copyright or intellectual property rights
• Generating content intended to harass, abuse, or discriminate
• Creating videos for illegal purposes or activities

Bias, Risks, and Limitations

Known Limitations

Technical Limitations :

• Temporal Consistency : May produce flickering or inconsistent motion in long sequences
• Fine Details : Small objects or intricate textures may lack detail or consistency
• Physical Realism : Generated physics may not always follow real-world rules (gravity, momentum, etc.)
• Text Rendering : Cannot reliably render readable text within generated videos
• Face Quality : Faces may show artifacts or unnatural movements (mitigated by face-naturalizer LoRA)
• Memory Requirements : High VRAM requirements limit accessibility (12-32GB depending on precision)

Content Limitations :

• Training data biases may affect representation of diverse demographics, cultures, and scenarios
• May struggle with uncommon objects, rare scenarios, or niche content
• Camera control may not always precisely match intended movements
• Generated content may reflect biases present in training data

Risks and Mitigations

Misuse Risks :

• Deepfakes and Misinformation : Model could be used to create deceptive content
  • Mitigation : Implement content authentication, watermarking, and usage monitoring
• Copyright Infringement : May generate content similar to copyrighted material
  • Mitigation : Avoid training on copyrighted data, implement content filtering
• Harmful Content : Could generate disturbing or inappropriate content
  • Mitigation : Implement safety filters, content moderation, and responsible use guidelines

Ethical Considerations :

• Users should obtain appropriate permissions before generating videos of identifiable individuals
• Generated content should be clearly labeled as AI-generated to prevent deception
• Consider environmental impact of compute-intensive inference
• Respect privacy, consent, and intellectual property rights

Recommendations

• Implement content moderation and safety filters in production deployments
• Add visible/invisible watermarks to identify AI-generated content
• Provide clear disclaimers that content is AI-generated
• Monitor for misuse and implement usage policies
• Consider accessibility trade-offs when selecting model precision
• Validate outputs for unintended biases or harmful content before distribution

Training Details

Training Data

Training data details are not publicly available. Typical video diffusion models are trained on:

• Large-scale video datasets with diverse content
• Text-video pairs for caption conditioning
• Image-video pairs for image-to-video tasks

Note : Specific training dataset information should be obtained from the original model authors.

Training Procedure

Training Hyperparameters (typical for models of this scale):

• Architecture: Diffusion transformer with 14B parameters
• Precision formats: FP16, FP8, GGUF quantization
• Video VAE: Separate encoder/decoder for latent compression
• LoRA adapters: Rank-16 to rank-64 for camera control

Noise Schedules :

• High-noise models : Greater noise variance for creative generation
• Low-noise models : Lower noise variance for faithful reproduction

Compute Infrastructure

Inference Requirements :

• FP16 : 24-32GB VRAM (NVIDIA A6000, RTX 4090, A5000)
• FP8 : 16-20GB VRAM (NVIDIA RTX 4080, RTX 3090)
• GGUF : 12-16GB VRAM (NVIDIA RTX 4070 Ti, RTX 3080)

Environmental Impact

Video generation models require significant computational resources:

• Model Size : 142GB total (including all variants and LoRAs)
• Inference : High power consumption during video generation (100-450W depending on GPU)
• Carbon Footprint : Varies by energy source and usage patterns

Recommendations for Reducing Impact :

• Use quantized models (FP8, GGUF) when quality trade-offs are acceptable
• Batch process multiple requests to amortize overhead
• Use energy-efficient hardware and renewable energy sources when possible
• Consider carbon offset for production deployments

License

Please check the original WAN 2.2 model repository for specific license terms and usage restrictions. This repository uses the "other" license tag pending clarification of the original license.

Citation

If you use WAN 2.2 in your research or applications, please cite the original model repository.

BibTeX (template):

@misc{wan22,
  title={WAN 2.2: Image-to-Video and Text-to-Video Generation},
  author={[Original Authors]},
  year={2024},
  howpublished={\url{https://huggingface.co/[original-repo]}},
}

Model Card Authors

This model card was created by the repository maintainer based on available model information and standard Hugging Face model card guidelines.

Model Card Contact

For questions about this model card or repository, please open an issue in the repository or contact the original model authors.

Troubleshooting

Out of Memory Errors :

• Switch to FP8 or GGUF quantized models
• Reduce num_frames or batch size
• Enable CPU offloading: pipe.enable_model_cpu_offload()

Quality Issues :

• Try both high-noise and low-noise variants
• Adjust LoRA weights (0.5-1.0 range)
• Increase num_inference_steps (50-100)

Slow Generation :

• Use FP8 models instead of FP16
• Enable xformers: pipe.enable_xformers_memory_efficient_attention()
• Reduce inference steps for testing

Support

For issues, questions, or contributions, please refer to the main Hugging Face model repository.

---

Last Updated : October 2025 Model Version : WAN 2.2 Repository Type : Model Weights Storage