BiliSakura / ZoomLDM-CDM-brca

BiliSakura/ZoomLDM-CDM-brca

Diffusers-style wrapped CDM (DiT) checkpoint for BRCA, converted from ZoomLDM cdm_dit training outputs.

Model Description

• Architecture: DiT-B style conditioning diffusion model (CDM)
• Domain: BRCA conditioning space used by ZoomLDM
• Output: conditioning tokens/embeddings ( (B, 512, 65) )
• Format: custom diffusers pipeline ( pipeline.py )

Intended Use

Use this model to sample BRCA conditioning embeddings that can be consumed by downstream ZoomLDM workflows.

Out-of-Scope Use

• Not a complete pixel-space generator by itself.
• Not intended for clinical or diagnostic use.
• Not validated for non-BRCA domains without adaptation.

Files

• pipeline.py : custom DiffusionPipeline implementation ( CDMDiTPipeline )
• model_index.json : diffusers metadata
• cdm/ : active model weights/config used by pipeline
• scheduler/ : DDIM scheduler config
• model_raw.safetensors : non-EMA training weights (optional)
• optimizer.pt : optimizer state (optional)
• config.json : conversion metadata

Usage

import torch
from diffusers import DiffusionPipeline

pipe = DiffusionPipeline.from_pretrained(
    "BiliSakura/ZoomLDM-CDM-brca",
    custom_pipeline="pipeline.py",
    trust_remote_code=True,
).to("cuda")

out = pipe(
    batch_size=2,
    magnification=torch.tensor([0, 0], device="cuda"),  # class labels 0..7
    num_inference_steps=50,
    guidance_scale=1.0,
)

samples = out.samples  # (B, 512, 65)

Limitations

• Produces conditioning embeddings, not final images.
• Requires correct class/magnification label conventions.
• Inherits data biases and quality limits from the original training data.

Citation

@InProceedings{Yellapragada_2025_CVPR,
  author = {Yellapragada, Srikar and Graikos, Alexandros and Triaridis, Kostas and Prasanna, Prateek and Gupta, Rajarsi and Saltz, Joel and Samaras, Dimitris},
  title = {ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation},
  booktitle = {Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR)},
  month = {June},
  year = {2025},
  pages = {23453-23463}
}

@inproceedings{Peebles2023DiT,
  title={Scalable Diffusion Models with Transformers},
  author={Peebles, William and Xie, Saining},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2023}
}