Sponsored by
Vmake Labs Video Enhancer
m-a-p / MERT-v2-FullSong
Model's Last Updated: September 30 2026
feature-extraction
Introduction of MERT-v2-FullSong
Model Details of MERT-v2-FullSong
🤗 MERT-v2-FullSong
Music representations with full-song context
24 kHz mono · 632M parameters · 24 layers · 1,024 dimensions · 25 Hz
🎵 YuE2 project · 🚀 Quick start · 📊 MARBLE · 🎛️ Layer guide · 📚 Citation
MERT-v2-FullSong is a bidirectional music encoder adapted to complete songs lasting 30–360 seconds. Extract general-purpose music representations at the frame or recording level. Load with standard Hugging Face Transformers, using the familiar MERT workflow.
It continues pretraining from MERT-v2-30s and preserves the same feature interface.
MERT-v2-30s uses the bidirectional backbone in (B); MERT-v2-FullSong continues through the full-song branch in (C). The causal branch is used for YuE2 tokenization.
🚀 Quick start
Install the matching PyTorch packages:
python -m pip install torch==2.6.0 torchaudio==2.6.0 transformers==4.53.2 huggingface-hub safetensors soundfile
import soundfile as sf
import torch
import torchaudio.functional as AF
from transformers import AutoFeatureExtractor, AutoModel
repo = "m-a-p/MERT-v2-FullSong"
device = "cuda" if torch.cuda.is_available() else "cpu"
processor = AutoFeatureExtractor.from_pretrained(repo, trust_remote_code=True)
model = AutoModel.from_pretrained(repo, trust_remote_code=True).eval().to(device)
audio, sr = sf.read("music.wav", dtype="float32", always_2d=True)
audio = audio[:30 * sr].mean(axis=1) # First 30 seconds, mixed to mono.
waveform = AF.resample(torch.from_numpy(audio), sr, processor.sampling_rate)
inputs = processor(
waveform.numpy(), sampling_rate=processor.sampling_rate, return_tensors="pt",
).to(device)
with torch.inference_mode():
output = model(**inputs, output_hidden_states=True)
frames = output.last_hidden_state # [batch, frames, 1024], 25 Hz
layers = output.hidden_states # 24 tensors, one per block
mask = output.feature_attention_mask[..., None]
embedding = (frames * mask).sum(1) / mask.sum(1).clamp_min(1) # [batch, 1024]
hidden_states[0]
is block 1;
hidden_states[23]
is block 24. Remove the audio slice to process a complete song.
📊 MARBLE
Reported frozen-encoder results on MARBLE . All scores are multiplied by 100; higher is better. Bold marks the best displayed value in each column, including ties.
Baseline scores and parameter counts are reproduced from Tables III and V of PupuJEPA ; the baselines were not rerun for this release. Evaluation protocols may differ across sources.
General music understanding. MTT = MagnaTagATune; key = GiantSteps refined key accuracy; genre and beat = GTZAN; valence and arousal = EmoMusic. ROC = ROC-AUC; AP = average precision.
| Model | Params | MTT ROC | MTT AP | Key acc. | Genre acc. | Beat F1 | Valence R² | Arousal R² |
|---|---|---|---|---|---|---|---|---|
| MERT-Large | 330M | 90.6 | 37.9 | 64.1 | 77.6 | 86.8 | 56.7 | 76.1 |
| Dasheng-1.2B | 1.2B | 91.5 | 40.4 | 58.0 | 81.4 | 87.7 | 57.4 | 75.0 |
| MuQ | 310M | 90.5 | 38.5 | 63.2 | 83.8 | 90.1 | 58.3 | 76.4 |
| MusicFM | 330M | 90.9 | 38.3 | 63.0 | 84.1 | 90.2 | 57.2 | 74.4 |
| AudioMAE++ | 307M | 91.2 | 39.5 | 61.7 | 80.3 | 90.0 | 59.0 | 75.7 |
| MATPAC++ | 307M | 90.6 | 38.2 | 63.7 | 81.4 | 90.1 | 57.8 | 74.7 |
| A-JEPA | 307M | 91.0 | 39.2 | 65.0 | 83.8 | 90.0 | 57.4 | 74.8 |
| PupuJEPA-Large | 307M | 91.7 | 40.8 | 66.1 | 86.9 | 91.0 | 62.5 | 76.8 |
| PupuJEPA-Huge | 632M | 91.3 | 39.7 | 64.8 | 85.9 | 90.5 | 62.0 | 78.5 |
| MERT-v2-30s | 632M | 91.91 | 41.29 | 66.97 | 91.72 | 90.59 | 63.23 | 80.01 |
| MERT-v2-FullSong | 632M | 91.74 | 41.20 | 67.05 | 90.69 | 90.57 | 63.52 | 78.14 |
MTG-Jamendo tagging. Mood denotes mood/theme tags.
| Model | Params | Instrument ROC | Instrument AP | Mood ROC | Mood AP | Genre ROC | Genre AP | Top-50 ROC | Top-50 AP |
|---|---|---|---|---|---|---|---|---|---|
| MERT-Large | 330M | 75.5 | 18.8 | 75.3 | 13.5 | 86.1 | 18.0 | 82.6 | 29.1 |
| Dasheng-1.2B | 1.2B | 75.0 | 19.0 | 76.1 | 15.5 | 85.5 | 18.8 | 82.4 | 29.6 |
| MuQ | 310M | 74.8 | 19.1 | 73.7 | 13.2 | 85.4 | 19.1 | 83.0 | 30.2 |
| MusicFM | 330M | 74.6 | 18.5 | 74.9 | 14.1 | 85.3 | 19.4 | 81.9 | 29.7 |
| AudioMAE++ | 307M | 77.1 | 19.9 | 75.6 | 14.0 | 86.3 | 18.9 | 83.1 | 31.1 |
| MATPAC++ | 307M | 77.2 | 19.7 | 75.1 | 14.1 | 85.7 | 19.6 | 82.5 | 30.2 |
| A-JEPA | 307M | 76.6 | 19.3 | 74.6 | 14.3 | 85.5 | 19.2 | 82.5 | 29.6 |
| PupuJEPA-Large | 307M | 78.4 | 21.2 | 76.2 | 15.3 | 86.1 | 20.1 | 82.8 | 30.5 |
| PupuJEPA-Huge | 632M | 77.6 | 20.5 | 75.9 | 14.7 | 85.9 | 20.1 | 83.1 | 30.7 |
| MERT-v2-30s | 632M | 80.27 | 22.89 | 79.44 | 16.68 | 88.01 | 21.22 | 84.18 | 32.17 |
| MERT-v2-FullSong | 632M | 80.27 | 23.51 | 78.74 | 15.74 | 87.98 | 20.66 | 84.13 | 31.62 |
Supplemental: Chords1217 frame accuracy is 78.48 for MERT-v2-30s and 77.79 for MERT-v2-FullSong.
HF reproduction verified: Both models reproduce all ten MARBLE tasks with the fixed evaluation settings. Scores and best settings .
🎛️ Layer guide
Recommended probe settings with the encoder frozen. L1 =
hidden_states[0]
; All-layer MLP uses all 24 layers.
| Task / dataset | MERT-v2-30s layer | Probe LR | MERT-v2-FullSong layer | Probe LR |
|---|---|---|---|---|
| Genre · GTZAN | L23 | 5e-3 | L24 | 5e-4 |
| Beat · GTZAN | L21 | 1e-3 | L23 | 1e-3 |
| Key · GiantSteps | L4 | 1e-3 | L23 | 1e-3 |
| Emotion · EmoMusic | All-layer MLP | 5e-5 | L24 | 5e-4 |
| Chords · Chords1217 | All-layer MLP | 1e-4 | All-layer MLP | 5e-4 |
| Tagging · MagnaTagATune | L22 | 1e-3 | L23 | 1e-3 |
| Instrument · MTG-Jamendo | L14 | 1e-3 | L12 | 1e-3 |
| Mood/theme · MTG-Jamendo | L16 | 1e-3 | L13 | 1e-3 |
| Genre · MTG-Jamendo | L19 | 1e-3 | L16 | 1e-3 |
| Top-50 · MTG-Jamendo | L13 | 1e-3 | L22 | 1e-3 |
📚 Citation
Technical report coming soon. For now, please cite MERT (ICLR 2024) and MARBLE (NeurIPS 2023) when using MERT-v2 or its benchmark results in your research.
@inproceedings{li2024mert,
title = {MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training},
author = {Li, Yizhi and Yuan, Ruibin and Zhang, Ge and Ma, Yinghao and Chen, Xingran and Yin, Hanzhi and Xiao, Chenghao and Lin, Chenghua and Ragni, Anton and Benetos, Emmanouil and Gyenge, Norbert and Dannenberg, Roger and Liu, Ruibo and Chen, Wenhu and Xia, Gus and Shi, Yemin and Huang, Wenhao and Wang, Zili and Guo, Yike and Fu, Jie},
booktitle = {International Conference on Learning Representations},
year = {2024},
url = {https://proceedings.iclr.cc/paper_files/paper/2024/hash/33dffa2e3d2ab74a783d1a8c292f66d9-Abstract-Conference.html}
}
@inproceedings{yuan2023marble,
title = {MARBLE: Music Audio Representation Benchmark for Universal Evaluation},
author = {Yuan, Ruibin and Ma, Yinghao and Li, Yizhi and Zhang, Ge and Chen, Xingran and Yin, Hanzhi and Zhuo, Le and Liu, Yiqi and Huang, Jiawen and Tian, Zeyue and Deng, Binyue and Wang, Ningzhi and Lin, Chenghua and Benetos, Emmanouil and Ragni, Anton and Gyenge, Norbert and Dannenberg, Roger and Chen, Wenhu and Xia, Gus and Xue, Wei and Liu, Si and Wang, Shi and Liu, Ruibo and Guo, Yike and Fu, Jie},
booktitle = {Advances in Neural Information Processing Systems},
volume = {36},
pages = {39626--39647},
year = {2023},
doi = {10.52202/075280-1722},
url = {https://proceedings.neurips.cc/paper_files/paper/2023/hash/7cbeec46f979618beafb4f46d8f39f36-Abstract-Datasets_and_Benchmarks.html}
}
Weights: CC BY-NC 4.0 . Dependency notices .
YuE2 family: Song generation · Audio decoder · Benchmark decoder .
Runs of m-a-p MERT-v2-FullSong on huggingface.co
20.7K
Total runs
0
24-hour runs
148
3-day runs
1.2K
7-day runs
18.8K
30-day runs
More Information About MERT-v2-FullSong huggingface.co Model
Url of MERT-v2-FullSong
MERT-v2-FullSong huggingface.co Url
Provider of MERT-v2-FullSong huggingface.co
Other API from m-a-p
Total runs:
4.1K
Run Growth:
-4.7K
Growth Rate:
-115.68%
Total runs:
3.6K
Run Growth:
2.8K
Growth Rate:
76.20%
Total runs:
2.3K
Run Growth:
946
Growth Rate:
41.67%
Total runs:
899
Run Growth:
314
Growth Rate:
34.93%
Total runs:
887
Run Growth:
176
Growth Rate:
19.84%
Total runs:
878
Run Growth:
693
Growth Rate:
78.93%
Total runs:
847
Run Growth:
312
Growth Rate:
36.84%
Total runs:
712
Run Growth:
-1.6K
Growth Rate:
-223.31%
Total runs:
675
Run Growth:
457
Growth Rate:
67.70%
Total runs:
611
Run Growth:
436
Growth Rate:
71.36%
Total runs:
525
Run Growth:
-382
Growth Rate:
-72.76%
Total runs:
511
Run Growth:
217
Growth Rate:
42.47%
Total runs:
480
Run Growth:
318
Growth Rate:
66.25%
Total runs:
416
Run Growth:
200
Growth Rate:
48.08%
Total runs:
404
Run Growth:
387
Growth Rate:
95.79%
Total runs:
362
Run Growth:
-80
Growth Rate:
-22.10%
Total runs:
247
Run Growth:
70
Growth Rate:
28.34%
Total runs:
189
Run Growth:
15
Growth Rate:
7.94%
Total runs:
167
Run Growth:
68
Growth Rate:
40.72%
Total runs:
162
Run Growth:
33
Growth Rate:
20.37%
Total runs:
112
Run Growth:
7
Growth Rate:
6.25%
Total runs:
112
Run Growth:
95
Growth Rate:
84.82%
Total runs:
102
Run Growth:
67
Growth Rate:
65.69%
Total runs:
97
Run Growth:
55
Growth Rate:
56.70%
Total runs:
89
Run Growth:
25
Growth Rate:
28.09%
Total runs:
70
Run Growth:
26
Growth Rate:
37.14%
Total runs:
64
Run Growth:
48
Growth Rate:
75.00%
Total runs:
64
Run Growth:
33
Growth Rate:
51.56%
Total runs:
59
Run Growth:
38
Growth Rate:
64.41%
Total runs:
56
Run Growth:
33
Growth Rate:
58.93%
Total runs:
53
Run Growth:
32
Growth Rate:
60.38%
Total runs:
52
Run Growth:
32
Growth Rate:
61.54%
Total runs:
52
Run Growth:
27
Growth Rate:
51.92%
Total runs:
52
Run Growth:
28
Growth Rate:
53.85%
Total runs:
51
Run Growth:
27
Growth Rate:
52.94%
Total runs:
48
Run Growth:
27
Growth Rate:
56.25%
Total runs:
48
Run Growth:
25
Growth Rate:
52.08%
Total runs:
47
Run Growth:
26
Growth Rate:
55.32%
Total runs:
47
Run Growth:
29
Growth Rate:
61.70%
Total runs:
47
Run Growth:
24
Growth Rate:
51.06%
Total runs:
46
Run Growth:
25
Growth Rate:
54.35%
Total runs:
46
Run Growth:
25
Growth Rate:
54.35%
Total runs:
46
Run Growth:
26
Growth Rate:
56.52%
Total runs:
45
Run Growth:
28
Growth Rate:
62.22%
Total runs:
45
Run Growth:
-12
Growth Rate:
-26.67%
Total runs:
44
Run Growth:
-21
Growth Rate:
-47.73%
Total runs:
43
Run Growth:
22
Growth Rate:
51.16%
Total runs:
39
Run Growth:
17
Growth Rate:
43.59%
Total runs:
38
Run Growth:
15
Growth Rate:
39.47%
Total runs:
38
Run Growth:
19
Growth Rate:
50.00%
Total runs:
36
Run Growth:
29
Growth Rate:
80.56%
Total runs:
35
Run Growth:
17
Growth Rate:
48.57%
Total runs:
35
Run Growth:
17
Growth Rate:
48.57%
Total runs:
34
Run Growth:
12
Growth Rate:
35.29%
Total runs:
34
Run Growth:
7
Growth Rate:
20.59%
Total runs:
33
Run Growth:
11
Growth Rate:
33.33%
Total runs:
33
Run Growth:
-2
Growth Rate:
-6.06%
Total runs:
33
Run Growth:
10
Growth Rate:
30.30%
Total runs:
33
Run Growth:
-4
Growth Rate:
-12.12%
Total runs:
32
Run Growth:
12
Growth Rate:
37.50%
Total runs:
32
Run Growth:
15
Growth Rate:
46.88%
Total runs:
31
Run Growth:
8
Growth Rate:
25.81%
Total runs:
31
Run Growth:
11
Growth Rate:
35.48%
Total runs:
29
Run Growth:
10
Growth Rate:
34.48%
Total runs:
27
Run Growth:
8
Growth Rate:
29.63%
Total runs:
26
Run Growth:
7
Growth Rate:
26.92%
Total runs:
26
Run Growth:
-5
Growth Rate:
-19.23%
Total runs:
25
Run Growth:
-7
Growth Rate:
-28.00%
Total runs:
25
Run Growth:
-2
Growth Rate:
-8.00%
Total runs:
25
Run Growth:
-2
Growth Rate:
-8.00%
Total runs:
25
Run Growth:
12
Growth Rate:
48.00%
Total runs:
24
Run Growth:
-8
Growth Rate:
-33.33%
Total runs:
24
Run Growth:
-4
Growth Rate:
-16.67%
Total runs:
23
Run Growth:
5
Growth Rate:
21.74%
Total runs:
22
Run Growth:
10
Growth Rate:
45.45%
Total runs:
21
Run Growth:
10
Growth Rate:
47.62%