Sponsored by
Vmake Labs Video Enhancer
sayakpaul / show-1-base
Model's Last Updated: October 23 2023
text-to-video
Introduction of show-1-base
Model Details of show-1-base
show-1-base
Pixel-based VDMs can generate motion accurately aligned with the textual prompt but typically demand expensive computational costs in terms of time and GPU memory, especially when generating high-resolution videos. Latent-based VDMs are more resource-efficient because they work in a reduced-dimension latent space. But it is challenging for such small latent space (e.g., 64×40 for 256×160 videos) to cover rich yet necessary visual semantic details as described by the textual prompt.
To marry the strength and alleviate the weakness of pixel-based and latent-based VDMs, we introduce Show-1 , an efficient text-to-video model that generates videos of not only decent video-text alignment but also high visual quality.
Model Details
This is the base model of Show-1 that generates videos with 8 keyframes at a resolution of 64x40. The model is finetuned from DeepFloyd/IF-I-L-v1.0 on the WebVid-10M and InternVid dataset.
- Developed by: Show Lab, National University of Singapore
- Model type: pixel- and latent-based cascaded text-to-video diffusion model
- Cascade stage: base (keyframe generation)
- Finetuned from model: DeepFloyd/IF-I-L-v1.0
- License: Creative Commons Attribution Non Commercial 4.0
- Resources for more information: GitHub , Website , arXiv
Usage
Clone the GitHub repository and install the requirements:
git clone https://github.com/showlab/Show-1.git
pip install -r requirements.txt
Run the following command to generate a video from a text prompt. By default, this will automatically download all the model weights from huggingface.
python run_inference.py
You can also download the weights manually and change the
pretrained_model_path
in
run_inference.py
to run the inference.
git lfs install
# base
git clone https://huggingface.co/showlab/show-1-base
# interp
git clone https://huggingface.co/showlab/show-1-interpolation
# sr1
git clone https://huggingface.co/showlab/show-1-sr1
# sr2
git clone https://huggingface.co/showlab/show-1-sr2
Citation
If you make use of our work, please cite our paper.
@misc{zhang2023show1,
title={Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation},
author={David Junhao Zhang and Jay Zhangjie Wu and Jia-Wei Liu and Rui Zhao and Lingmin Ran and Yuchao Gu and Difei Gao and Mike Zheng Shou},
year={2023},
eprint={2309.15818},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Model Card Contact
This model card is maintained by David Junhao Zhang and Jay Zhangjie Wu . For any questions, please feel free to contact us or open an issue in the repository.
Runs of sayakpaul show-1-base on huggingface.co
3
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
-2
30-day runs
More Information About show-1-base huggingface.co Model
Url of show-1-base
show-1-base huggingface.co Url
Provider of show-1-base huggingface.co
Other API from sayakpaul
Total runs:
126
Run Growth:
49
Growth Rate:
38.89%
Total runs:
70
Run Growth:
-11
Growth Rate:
-21.15%
Total runs:
56
Run Growth:
24
Growth Rate:
42.11%
Total runs:
41
Run Growth:
12
Growth Rate:
26.09%
Total runs:
37
Run Growth:
6
Growth Rate:
16.22%
Total runs:
37
Run Growth:
-9
Growth Rate:
-24.32%
Total runs:
33
Run Growth:
-4
Growth Rate:
-12.12%
Total runs:
32
Run Growth:
8
Growth Rate:
25.00%
Total runs:
32
Run Growth:
6
Growth Rate:
18.75%
Total runs:
30
Run Growth:
0
Growth Rate:
0.00%
Total runs:
30
Run Growth:
0
Growth Rate:
0.00%
Total runs:
24
Run Growth:
-30
Growth Rate:
-125.00%
Total runs:
23
Run Growth:
9
Growth Rate:
39.13%
Total runs:
20
Run Growth:
18
Growth Rate:
90.00%
Total runs:
20
Run Growth:
8
Growth Rate:
40.00%
Total runs:
15
Run Growth:
-9
Growth Rate:
-60.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
13
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
12
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
0
Growth Rate:
0.00%
Total runs:
11
Run Growth:
-17
Growth Rate:
-154.55%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
-9
Growth Rate:
-90.00%
Total runs:
10
Run Growth:
8
Growth Rate:
80.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
0
Growth Rate:
0.00%
Total runs:
10
Run Growth:
-2
Growth Rate:
-20.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
3
Growth Rate:
33.33%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
0
Growth Rate:
0.00%
Total runs:
9
Run Growth:
8
Growth Rate:
88.89%
Total runs:
8
Run Growth:
4
Growth Rate:
50.00%
Total runs:
8
Run Growth:
1
Growth Rate:
12.50%
Total runs:
8
Run Growth:
0
Growth Rate:
0.00%
Total runs:
7
Run Growth:
0
Growth Rate:
0.00%
Total runs:
6
Run Growth:
-1
Growth Rate:
-16.67%
Total runs:
6
Run Growth:
0
Growth Rate:
0.00%
Total runs:
6
Run Growth:
1
Growth Rate:
16.67%
Total runs:
6
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
-2
Growth Rate:
-40.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
5
Run Growth:
3
Growth Rate:
60.00%
Total runs:
5
Run Growth:
0
Growth Rate:
0.00%
Total runs:
4
Run Growth:
-1
Growth Rate:
-25.00%
Total runs:
4
Run Growth:
3
Growth Rate:
75.00%
Total runs:
4
Run Growth:
0
Growth Rate:
0.00%
Total runs:
4
Run Growth:
-3
Growth Rate:
-75.00%
Total runs:
3
Run Growth:
1
Growth Rate:
33.33%
Total runs:
3
Run Growth:
-3
Growth Rate:
-100.00%
Total runs:
2
Run Growth:
-6
Growth Rate:
-300.00%
Total runs:
2
Run Growth:
1
Growth Rate:
50.00%
Total runs:
2
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1
Run Growth:
0
Growth Rate:
0.00%
Total runs:
1
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%