Total runs:
209.8K
Run Growth:
96.7K
Growth Rate:
46.09%
Sponsored by
APIMaster
ByteDance / shot2story
Model's Last Updated: June 17 2024
visual-question-answering
Introduction of shot2story
Model Details of shot2story
Shot2Story: A New Benchmark for Comprehensive Understanding of Multi-shot Videos
- Repository: Shot2Story
- Paper: 2312.10300
- Point of Contact: mailto: Mingfei Han
Training Dataset
Please download the multi-shot videos here .
We are excited to release a new video-text benchmark for multi-shot video understanding. This release contains a 134k version of our dataset. It includes detailed long summaries (human annotated + GPTV generated) for 134k videos and shot captions (human annotated) for 188k video shots. Please check the dataset here .
Models
We are releasing the checkpoints trained with our Shot2Story-20K and Shot2Story-134K .
-
{20k,134k}-version/sum_shot_best_epoch.pth:
Model tuned on our multi-shot summary data. Used in the config files
ckpt. -
{20k,134k}-version/shot_av_best_epoch.pth:
Model trained on our single-shot caption data. Used in the config files
ckpt. - transnetv2-pytorch-weights.pth: Checkpoint used for automatic shot detection method, which is used in the Bot demo. Please following the original license of the TransNetv2.
- BLIP.cache.tar: Cached checkpoints for training, testing and offline demos. This is only to ease the usage case that servers can't access huggingface. Please be restriected the original license to the different models.
License
Our text annotations are licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License . They are available strictly for non-commercial research.
Users must refer to HD-VILA-100M for original video access. By downloading our annotations, you agree to these terms. Respect for video copyright holders is paramount. Ensure your use of the videos aligns with the original source's terms.
Citation
If you find our work useful for your research, please consider citing the paper
@misc{han2023shot2story20k,
title={Shot2Story20K: A New Benchmark for Comprehensive Understanding of Multi-shot Videos},
author={Mingfei Han and Linjie Yang and Xiaojun Chang and Heng Wang},
year={2023},
eprint={2312.10300},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Runs of ByteDance shot2story on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About shot2story huggingface.co Model
Url of shot2story
shot2story huggingface.co Url
Provider of shot2story huggingface.co
Other API from ByteDance
Updated:June 12 2025
Total runs:
91.2K
Run Growth:
3.9K
Growth Rate:
4.28%
Total runs:
85.1K
Run Growth:
54.4K
Growth Rate:
63.97%
Total runs:
21.1K
Run Growth:
6.3K
Growth Rate:
29.99%
Total runs:
14.7K
Run Growth:
4.7K
Growth Rate:
31.95%
Total runs:
12.5K
Run Growth:
-1.4K
Growth Rate:
-11.53%
Total runs:
1.1K
Run Growth:
1.1K
Growth Rate:
94.01%
Total runs:
938
Run Growth:
-76
Growth Rate:
-8.10%
Total runs:
887
Run Growth:
507
Growth Rate:
57.16%
Total runs:
392
Run Growth:
328
Growth Rate:
83.67%
Total runs:
219
Run Growth:
163
Growth Rate:
74.43%
Total runs:
187
Run Growth:
50
Growth Rate:
26.74%
Total runs:
141
Run Growth:
25
Growth Rate:
17.73%
Total runs:
122
Run Growth:
90
Growth Rate:
73.77%
Total runs:
106
Run Growth:
-3
Growth Rate:
-2.83%
Total runs:
59
Run Growth:
25
Growth Rate:
42.37%
Total runs:
47
Run Growth:
22
Growth Rate:
46.81%
Total runs:
13
Run Growth:
1
Growth Rate:
7.69%
Total runs:
13
Run Growth:
8
Growth Rate:
61.54%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%