Total runs:
199.9K
Run Growth:
124.2K
Growth Rate:
63.53%
Sponsored by
APIMart
ByteDance / Attention2Probability
Model's Last Updated: August 27 2025
automatic-speech-recognition
Introduction of Attention2Probability
Model Details of Attention2Probability
Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System
Attention2Probability (A2P) is a lightweight intervention scheme for speech terminology. The core approach is to use the cross-attention mechanism to retrieve the terms that may appear in the audio and add these terms to the prompt of the llm to complete the term intervention.
News
- [2025-08-27] We have released the train and infer code for A2P.
Structure
The overall architecture of Attention2Probability. Audio features are extracted and then fed into a cross-attention retriever, which retrieves the Top-k terms with the highest probability of occurrence within the audio. These retrieved terms are concatenated with the prompt. Finally, the prompt and the audio features are jointly input into the speech large language model.
Installation
A2P is implemented based on the open-source toolkit accelerate
pip3 install -r requirements.txt
Training
-
Download the data to
/pathtodata. It's important to change your audio path in json. -
Download the model to
/path/pretrained-modelh. Your can also download Qwen2-Audio-Instruction and split it to the audio_tower, projector and embedding. - Running with ```bash ./retriever/train.sh`` in A100-SXM-80GB.
- For the dataset configuration, the phrase_type parameter can be adjusted to specify either word-level or phrase-level granularity. It should be noted that models for Chinese are generally trained only at the phrase-level, as word-level granularity is nonsensical for the Chinese language.
Inference
- Same as Training: 1-2 .
-
Download the ckpt to
ckpt. -
Running with
python3 ./infer/infer.py --config ./infer/infer_configin A100-SXM-80GB. Now you can change the setting in theinfer_config.json. Enjoy yourself !
Citation
If you find A2P useful, please cite the paper:
@inproceedings{
dy2025attention,
title={{Attention2Probability: Attention-Driven Terminology Probability Estimation for Robust Speech-to-Text System},
author={Yangfan Du, Jun Zhang, Bin Wang, Jin Qiu, Lu Huang, Yuan Ge, Xiaoqian Liu, Tong Xiao, Jingbo Zhu},
}
Runs of ByteDance Attention2Probability on huggingface.co
0
Total runs
0
24-hour runs
0
3-day runs
0
7-day runs
0
30-day runs
More Information About Attention2Probability huggingface.co Model
Url of Attention2Probability
Attention2Probability huggingface.co Url
Provider of Attention2Probability huggingface.co
Other API from ByteDance
Updated:April 03 2024
Total runs:
45.4K
Run Growth:
-13.4K
Growth Rate:
-29.36%
Total runs:
20.1K
Run Growth:
-169.8K
Growth Rate:
-843.45%
Total runs:
10.6K
Run Growth:
4.3K
Growth Rate:
40.68%
Total runs:
10.4K
Run Growth:
2.5K
Growth Rate:
23.87%
Total runs:
3.7K
Run Growth:
9
Growth Rate:
0.24%
Total runs:
3.3K
Run Growth:
-731
Growth Rate:
-22.29%
Total runs:
1.3K
Run Growth:
-4.1K
Growth Rate:
-320.44%
Total runs:
694
Run Growth:
436
Growth Rate:
62.82%
Total runs:
625
Run Growth:
-1.3K
Growth Rate:
-208.48%
Total runs:
177
Run Growth:
35
Growth Rate:
19.77%
Total runs:
162
Run Growth:
46
Growth Rate:
28.40%
Total runs:
150
Run Growth:
40
Growth Rate:
26.67%
Total runs:
131
Run Growth:
-129
Growth Rate:
-98.47%
Total runs:
88
Run Growth:
-20
Growth Rate:
-22.73%
Total runs:
55
Run Growth:
9
Growth Rate:
16.36%
Total runs:
46
Run Growth:
-34
Growth Rate:
-73.91%
Total runs:
45
Run Growth:
5
Growth Rate:
11.63%
Total runs:
37
Run Growth:
28
Growth Rate:
73.68%
Total runs:
2
Run Growth:
-5
Growth Rate:
-500.00%
Total runs:
0
Run Growth:
0
Growth Rate:
0.00%