kernels-community / vllm-flash-attn3

huggingface.co
Total runs: 37.3K
24-hour runs: 0
7-day runs: 2.2K
30-day runs: 11.7K
Model's Last Updated: July 01 2026

Introduction of vllm-flash-attn3

Model Details of vllm-flash-attn3

vllm-flash-attn3

This is an implementation of Flash Attention 3 CUDA kernels with support for attention sinks. The attention sinks implementation was contributed to Flash Attention by the vLLM team . The transformers team packaged the implementation and pre-built it for use with the kernels library .

How to Use

When loading your model with transformers, provide this repository id as the source of the attention implementation:

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "<your model id on the Hub>"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map="auto",
    torch_dtype="auto",
+    # Flash Attention with Sinks
+    attn_implementation="kernels-community/vllm-flash-attn3”,
)

This will automatically resolve and download the appropriate code for your architecture. See more details in this post .

Credits

Runs of kernels-community vllm-flash-attn3 on huggingface.co

37.3K
Total runs
0
24-hour runs
1.1K
3-day runs
2.2K
7-day runs
11.7K
30-day runs

More Information About vllm-flash-attn3 huggingface.co Model

More vllm-flash-attn3 license Visit here:

https://choosealicense.com/licenses/apache-2.0

vllm-flash-attn3 huggingface.co

vllm-flash-attn3 huggingface.co is an AI model on huggingface.co that provides vllm-flash-attn3's model effect (), which can be used instantly with this kernels-community vllm-flash-attn3 model. huggingface.co supports a free trial of the vllm-flash-attn3 model, and also provides paid use of the vllm-flash-attn3. Support call vllm-flash-attn3 model through api, including Node.js, Python, http.

kernels-community vllm-flash-attn3 online free

vllm-flash-attn3 huggingface.co is an online trial and call api platform, which integrates vllm-flash-attn3's modeling effects, including api services, and provides a free online trial of vllm-flash-attn3, you can try vllm-flash-attn3 online for free by clicking the link below.

kernels-community vllm-flash-attn3 online free url in huggingface.co:

https://huggingface.co/kernels-community/vllm-flash-attn3

vllm-flash-attn3 install

vllm-flash-attn3 is an open source model from GitHub that offers a free installation service, and any user can find vllm-flash-attn3 on GitHub to install. At the same time, huggingface.co provides the effect of vllm-flash-attn3 install, users can directly use vllm-flash-attn3 installed effect in huggingface.co for debugging and trial. It also supports api for free installation.

vllm-flash-attn3 install url in huggingface.co:

https://huggingface.co/kernels-community/vllm-flash-attn3

Url of vllm-flash-attn3

Provider of vllm-flash-attn3 huggingface.co

kernels-community
ORGANIZATIONS

Other API from kernels-community