This repo contains model checkpoints for
Vamba-Qwen2-VL-7B
. Vamba is a hybrid Mamba-Transformer model that leverages cross-attention layers and Mamba-2 blocks for efficient hour-long video understanding.
The main computation overhead in the transformer-based LMMs comes from the quadratic complexity of the self-attention in the video tokens. To overcome this issue, we design a hybrid Mamba Transformer architecture to process text and video tokens differently. The key idea of our method is to split the expensive self-attention operation over the entire video and text token sequence into two more efficient components. Since video tokens typically dominate the sequence while text tokens remain few, we maintain the self-attention mechanism exclusively for the text tokens and eliminate it for the video tokens. Instead, we add cross-attention layers that use text tokens as queries and video tokens as keys and values. In the meantime, we propose employing Mamba blocks to effectively process the video tokens.
Citation
If you find our paper useful, please cite us with
coming soon
Runs of TIGER-Lab Vamba-Qwen2-VL-7B on huggingface.co
498
Total runs
0
24-hour runs
0
3-day runs
-11
7-day runs
110
30-day runs
More Information About Vamba-Qwen2-VL-7B huggingface.co Model
Vamba-Qwen2-VL-7B huggingface.co is an AI model on huggingface.co that provides Vamba-Qwen2-VL-7B's model effect (), which can be used instantly with this TIGER-Lab Vamba-Qwen2-VL-7B model. huggingface.co supports a free trial of the Vamba-Qwen2-VL-7B model, and also provides paid use of the Vamba-Qwen2-VL-7B. Support call Vamba-Qwen2-VL-7B model through api, including Node.js, Python, http.
Vamba-Qwen2-VL-7B huggingface.co is an online trial and call api platform, which integrates Vamba-Qwen2-VL-7B's modeling effects, including api services, and provides a free online trial of Vamba-Qwen2-VL-7B, you can try Vamba-Qwen2-VL-7B online for free by clicking the link below.
TIGER-Lab Vamba-Qwen2-VL-7B online free url in huggingface.co:
Vamba-Qwen2-VL-7B is an open source model from GitHub that offers a free installation service, and any user can find Vamba-Qwen2-VL-7B on GitHub to install. At the same time, huggingface.co provides the effect of Vamba-Qwen2-VL-7B install, users can directly use Vamba-Qwen2-VL-7B installed effect in huggingface.co for debugging and trial. It also supports api for free installation.