wangkanai / wan21-fp16

WAN 2.1 FP16 - Image-to-Video Generation with Camera Control

This repository contains WAN (Wan An) 2.1 models in full FP16 precision for advanced image-to-video generation with cinematic camera control capabilities. WAN 2.1 introduces LoRA adapters for precise camera movement control in generated videos.

Model Description

WAN 2.1 FP16 is a high-fidelity video generation model that transforms static images into dynamic videos with specialized camera control LoRAs. These LoRA adapters enable cinematic camera movements including rotation, arc shots, and drone perspectives. The FP16 precision provides maximum numerical accuracy and generation quality.

Model Components

Total Size : ~63GB

Diffusion Models (FP16 Full Precision)

Two image-to-video transformer models at different resolutions:

1. WAN 2.1 I2V 480p (14B parameters)

• File : diffusion_models/wan/wan21-i2v-480p-14b-fp16.safetensors
• Size : 31 GB
• Precision : FP16 (16-bit floating point)
• Resolution : 480p video generation
• Parameters : 14 billion
• Optimized for balanced quality and performance

2. WAN 2.1 I2V 720p (14B parameters)

• File : diffusion_models/wan/wan21-i2v-720p-14b-fp16.safetensors
• Size : 31 GB
• Precision : FP16 (16-bit floating point)
• Resolution : 720p video generation
• Parameters : 14 billion
• Maximum quality output with enhanced detail

VAE (Variational Autoencoder)

• File : vae/wan/wan21-vae.safetensors
• Size : 243 MB
• Precision : Standard (shared across all resolutions)
• High-quality VAE for encoding and decoding video latents

Camera Control LoRAs

Three specialized LoRA models for precise camera movement control (all rank-16, 343MB each):

1. Camera Rotation LoRA

• File : loras/wan/wan21-camera-rotation-rank16-v1.safetensors
• Size : 343 MB
• Rank : 16
• Enables smooth rotational camera movements around subjects

2. Camera Arc Shot LoRA

• File : loras/wan/wan21-camera-arcshot-rank16-v1.safetensors
• Size : 343 MB
• Rank : 16
• Creates cinematic arc/circular camera movements

3. Camera Drone Shot LoRA

• File : loras/wan/wan21-camera-drone-rank16-v1.safetensors
• Size : 343 MB
• Rank : 16
• Simulates aerial drone-style camera perspectives and movements

Hardware Requirements

• VRAM : 32GB+ recommended for 480p model, 40GB+ for 720p model (FP16 full precision)
• Disk Space : 63GB total (31GB per diffusion model + VAE + LoRAs)
• LoRA Overhead : 343MB per LoRA (minimal additional VRAM)
• System RAM : 32GB+ recommended for optimal performance
• GPU : High-end NVIDIA GPU (RTX 3090/4090, A6000, or better recommended)

Usage

Loading the Models

from diffusers import DiffusionPipeline, AutoencoderKL
import torch

# Load the 480p FP16 model (lower VRAM requirements)
pipe = DiffusionPipeline.from_single_file(
    "E:/huggingface/wan21-fp16/diffusion_models/wan/wan21-i2v-480p-14b-fp16.safetensors",
    torch_dtype=torch.float16,
    use_safetensors=True
)

# Or load the 720p FP16 model (maximum quality)
# pipe = DiffusionPipeline.from_single_file(
#     "E:/huggingface/wan21-fp16/diffusion_models/wan/wan21-i2v-720p-14b-fp16.safetensors",
#     torch_dtype=torch.float16,
#     use_safetensors=True
# )

# Load WAN 2.1 VAE
pipe.vae = AutoencoderKL.from_single_file(
    "E:/huggingface/wan21-fp16/vae/wan/wan21-vae.safetensors"
)

pipe.to("cuda")

Image-to-Video Generation with Camera Control

from PIL import Image

# Load input image
input_image = Image.open("path/to/your/image.jpg")

# Load desired camera control LoRA
pipe.load_lora_weights(
    "E:/huggingface/wan21-fp16/loras/wan/wan21-camera-rotation-rank16-v1.safetensors"
)

# Generate video from image with camera control
video = pipe(
    image=input_image,
    prompt="rotating camera around the subject",
    num_frames=24,
    num_inference_steps=50,
    guidance_scale=7.5
).frames[0]

# Save video
from diffusers.utils import export_to_video
export_to_video(video, "output_rotating.mp4", fps=8)

Camera Movement Prompting Tips

Rotation LoRA ( wan21-camera-rotation-rank16-v1.safetensors ):

• Prompts: "rotating camera", "camera circles around subject", "360-degree view"
• Effect: Orbital camera movement around the subject

Arc Shot LoRA ( wan21-camera-arcshot-rank16-v1.safetensors ):

• Prompts: "arc shot", "curved camera movement", "sweeping camera motion"
• Effect: Smooth curved dolly movements

Drone LoRA ( wan21-camera-drone-rank16-v1.safetensors ):

• Prompts: "aerial view", "drone shot", "bird's eye view", "flying camera"
• Effect: Aerial and elevated perspectives

Model Specifications

• Diffusion Model Architecture : 14B parameter transformer-based I2V model
• Precision : FP16 (16-bit floating point, full precision)
• VAE Architecture : Custom trained for video generation (standard precision)
• LoRA Rank : 16 (balanced between quality and efficiency)
• Format : SafeTensors (secure and efficient)
• Compatible With : diffusers library with FP16 support
• Resolutions : 480p and 720p variants available
• Quality : Maximum generation quality with full numerical precision

Combining Multiple LoRAs

# Load multiple camera control LoRAs
pipe.load_lora_weights(
    "E:/huggingface/wan21-fp16/loras/wan/wan21-camera-rotation-rank16-v1.safetensors",
    adapter_name="rotation"
)
pipe.load_lora_weights(
    "E:/huggingface/wan21-fp16/loras/wan/wan21-camera-drone-rank16-v1.safetensors",
    adapter_name="drone"
)

# Set different weights for each LoRA
pipe.set_adapters(["rotation", "drone"], adapter_weights=[0.7, 0.3])

# Generate with combined camera movements
video = pipe(
    image=input_image,
    prompt="rotating aerial drone shot of landscape",
    num_frames=24,
    num_inference_steps=50
).frames[0]

Installation

# Install required dependencies
pip install diffusers transformers accelerate safetensors torch

Requirements

• Python 3.8+
• PyTorch 2.0+
• diffusers
• transformers
• accelerate
• safetensors

Performance Tips

1. Resolution Selection : Use 480p model for faster generation, 720p for maximum quality
2. Memory Optimization : Enable gradient checkpointing and attention slicing for lower VRAM usage
3. Combining LoRAs : You can combine multiple camera LoRAs at lower weights for complex movements
4. Weight Adjustment : Adjust LoRA strength (typically 0.5-1.0) to control the intensity of camera effects
5. Quality vs Speed : FP16 provides best quality but requires more VRAM; consider FP8 for production
6. GPU Selection : Best performance on high-end GPUs with 32GB+ VRAM (A6000, RTX 4090, etc.)

Version Comparison

WAN 2.1 FP16 vs FP8 :

• FP16 (this repo) : Full precision, maximum quality, highest VRAM requirements (32GB+)
• FP8 : ~50% smaller model size, 40% VRAM savings, minor quality tradeoff, faster inference

WAN 2.1 vs WAN 2.2 :

• WAN 2.1 : Original camera control LoRAs (v1), 480p/720p resolutions
• WAN 2.2 : Enhanced camera controls (v2) + additional enhancement LoRAs (lighting, faces, actions)

Precision Selection Guide :

• Use FP16 for: Research, maximum quality requirements, archival work, GPU with 32GB+ VRAM
• Use FP8 for: Production deployment, VRAM-constrained systems, batch processing, inference optimization

For the best quality and numerical precision, FP16 is recommended when VRAM is not a constraint.

License

Check the specific license terms for WAN models. This may differ from standard open-source licenses.

Citation

If you use this model in your research or projects, please cite:

@software{wan2.1_fp16,
  title={WAN 2.1 FP16: Image-to-Video Generation with Camera Control},
  year={2024},
  note={Full precision FP16 models with camera control LoRAs for high-fidelity video generation}
}

Model Card Contact

For questions about WAN 2.1 FP16 models, refer to the official WAN model documentation and community resources.

Technical Notes

• FP16 Precision : 16-bit floating point (1 sign bit, 5-bit exponent, 10-bit mantissa)
• Quality : Maximum generation quality with full numerical precision
• VRAM Requirements : Higher than quantized variants but provides best results
• Compatibility : Requires PyTorch with FP16 support (all modern versions)
• Best Use Cases : Research, archival quality, professional production work

Related Resources

• WAN 2.1 FP8 - Quantized variants for efficient deployment (~50% smaller)
• WAN 2.2 Models - Enhanced camera controls and quality improvements
• WAN LightX2V Models - CFG step distillation adapters for faster generation

Advantages of FP16

• Maximum Quality : Full numerical precision for best generation quality
• Research Standard : Industry standard precision for research and development
• Flexibility : No quantization artifacts or edge cases
• Compatibility : Broad GPU and software support

Trade-offs

• VRAM Requirements : Requires 32GB+ VRAM for 480p, 40GB+ for 720p
• Storage : 2x larger than FP8 variants (63GB vs 33GB total)
• Inference Speed : Slightly slower than FP8 on GPUs with tensor core support
• Cost : Requires more expensive GPUs for deployment

---

Note : This is the full precision model repository optimized for maximum quality. For deployment scenarios with VRAM constraints, consider the FP8 quantized variants. Please use responsibly and in accordance with ethical AI guidelines.