Sponsored by
PoYo.AI
ATH-MaaS / Parrot-7B
Model's Last Updated: November 21 2024
image-text-to-text
Introduction of Parrot-7B
Model Details of Parrot-7B
Parrot-7B
Introduction
Welcome to Parrot, a novel method that utilizes textual guidance to drive visual token alignment at the language level. Parrot makes the visual tokens condition on diverse language inputs and uses Mixture-of-Experts (MoE) to promote the alignment of multilingual tokens. Moreover, considering the current lack of benchmarks for evaluating multilingual capabilities within the field, we collect and make available a Massive Multilingual Multimodal Benchmark which includes 6 languages, 15 categories, and 12,000 questions, named as MMMB. For a comprehensive introduction, please refer to Parrot Paper and Parrot GitHub .
Model
Parrot is a multilingual multimodal large language model. We provide our fully finetuned models below:
| Model | Base LLM | Vision Encoder | Stage | Download |
|---|---|---|---|---|
| Parrot-7B | Qwen-1.5-7B-Chat | CLIP-ViT-Large-patch14-336 | SFT | ckpt |
| Parrot-14B | Qwen-1.5-14B-Chat | CLIP-ViT-Large-patch14-336 | SFT | ckpt |
Performance
Quick Start
We provide a quick start demo in Parrot GitHub , which can be used as a template to run Parrot for inference.
- Before running the demo, please make sure you download the Parrot checkpoint and the Clip checkpoint .
-
Second, you should replace the paths in the
runner.py. - Finally, run the python file in your system.
Citation
If you find Parrot useful, please cite the paper
@article{sun2024parrot,
title={Parrot: Multilingual Visual Instruction Tuning},
author={Sun, Hai-Long and Zhou, Da-Wei and Li, Yang and Lu, Shiyin and Yi, Chao and Chen, Qing-Guo and Xu, Zhao and Luo, Weihua and Zhang, Kaifu and Zhan, De-Chuan and others},
journal={arXiv preprint arXiv:2406.02539},
year={2024}
}
License
The project is licensed under Apache License Version 2.0 and is restricted to uses that comply with the license agreements of Qwen and Clip.
Disclaimer
We used compliance-checking algorithms during the training process, to ensure the compliance of the trained model to the best of our ability. Due to the complexity of the data and the diversity of language model usage scenarios, we cannot guarantee that the model is completely free of copyright issues or improper content. If you believe anything infringes on your rights or generates improper content, please contact us, and we will promptly address the matter.
Runs of ATH-MaaS Parrot-7B on huggingface.co
29
Total runs
1
24-hour runs
2
3-day runs
5
7-day runs
18
30-day runs
More Information About Parrot-7B huggingface.co Model
Url of Parrot-7B
Parrot-7B huggingface.co Url
Provider of Parrot-7B huggingface.co
Other API from ATH-MaaS
Total runs:
8.3K
Run Growth:
-62.9K
Growth Rate:
-764.01%
Total runs:
4.7K
Run Growth:
-51.9K
Growth Rate:
-1087.27%
Total runs:
4.3K
Run Growth:
-12.0K
Growth Rate:
-282.72%
Total runs:
3.8K
Run Growth:
-4.0K
Growth Rate:
-108.37%
Total runs:
1.6K
Run Growth:
1.1K
Growth Rate:
69.79%
Total runs:
1.1K
Run Growth:
-14.2K
Growth Rate:
-1156.96%
Total runs:
854
Run Growth:
-22.6K
Growth Rate:
-2554.59%
Total runs:
698
Run Growth:
366
Growth Rate:
55.54%
Total runs:
495
Run Growth:
-9.4K
Growth Rate:
-1892.35%
Total runs:
269
Run Growth:
-291
Growth Rate:
-98.98%
Total runs:
268
Run Growth:
-336
Growth Rate:
-111.26%
Total runs:
231
Run Growth:
-2
Growth Rate:
-0.87%
Total runs:
112
Run Growth:
-66
Growth Rate:
-64.08%
Total runs:
62
Run Growth:
28
Growth Rate:
45.90%
Total runs:
54
Run Growth:
-41
Growth Rate:
-77.36%
Total runs:
49
Run Growth:
25
Growth Rate:
52.08%
Total runs:
39
Run Growth:
-86
Growth Rate:
-226.32%
Total runs:
37
Run Growth:
12
Growth Rate:
33.33%
Total runs:
28
Run Growth:
-30
Growth Rate:
-107.14%
Total runs:
24
Run Growth:
-2
Growth Rate:
-8.00%
Total runs:
19
Run Growth:
-6
Growth Rate:
-31.58%
Total runs:
12
Run Growth:
-10
Growth Rate:
-83.33%
Total runs:
11
Run Growth:
-2
Growth Rate:
-16.67%
Total runs:
11
Run Growth:
-2
Growth Rate:
-18.18%
Total runs:
10
Run Growth:
-3
Growth Rate:
-30.00%