The model architecture is the same as the
original HuBERT Base model
, which contains 12 transformer layers with 12 attention heads.
The model was trained using code from the
official repository
, and the detailed training configuration can be found in the same repository and the
original paper
.
Training
The model was trained on approximately 19,000 hours of following Japanese speech corpus ReazonSpeech v1.
import soundfile as sf
from transformers import AutoFeatureExtractor, AutoModel
model_name = "rinna/japanese-hubert-base"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name)
model.eval()
raw_speech_16kHz, sr = sf.read(audio_file)
inputs = feature_extractor(
raw_speech_16kHz,
return_tensors="pt",
sampling_rate=sr,
)
outputs = model(**inputs)
print(f"Input: {inputs.input_values.size()}") # [1, #samples]print(f"Output: {outputs.last_hidden_state.size()}") # [1, #frames, 768]
A fairseq checkpoint file can also be available
here
.
How to cite
@misc{rinna-japanese-hubert-base,
title = {rinna/japanese-hubert-base},
author = {Hono, Yukiya and Mitsui, Kentaro and Sawada, Kei},
url = {https://huggingface.co/rinna/japanese-hubert-base}
}
@inproceedings{sawada2024release,
title = {Release of Pre-Trained Models for the {J}apanese Language},
author = {Sawada, Kei and Zhao, Tianyu and Shing, Makoto and Mitsui, Kentaro and Kaga, Akio and Hono, Yukiya and Wakatsuki, Toshiaki and Mitsuda, Koh},
booktitle = {Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
month = {5},
year = {2024},
pages = {13898--13905},
url = {https://aclanthology.org/2024.lrec-main.1213},
note = {\url{https://arxiv.org/abs/2404.01657}}
}
References
@article{hsu2021hubert,
author = {Hsu, Wei-Ning and Bolte, Benjamin and Tsai, Yao-Hung Hubert and Lakhotia, Kushal and Salakhutdinov, Ruslan and Mohamed, Abdelrahman},
journal = {IEEE/ACM Transactions on Audio, Speech, and Language Processing},
title = {HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units},
year = {2021},
volume = {29},
pages = {3451-3460},
doi = {10.1109/TASLP.2021.3122291}
}
japanese-hubert-base huggingface.co is an AI model on huggingface.co that provides japanese-hubert-base's model effect (), which can be used instantly with this rinna japanese-hubert-base model. huggingface.co supports a free trial of the japanese-hubert-base model, and also provides paid use of the japanese-hubert-base. Support call japanese-hubert-base model through api, including Node.js, Python, http.
japanese-hubert-base huggingface.co is an online trial and call api platform, which integrates japanese-hubert-base's modeling effects, including api services, and provides a free online trial of japanese-hubert-base, you can try japanese-hubert-base online for free by clicking the link below.
rinna japanese-hubert-base online free url in huggingface.co:
japanese-hubert-base is an open source model from GitHub that offers a free installation service, and any user can find japanese-hubert-base on GitHub to install. At the same time, huggingface.co provides the effect of japanese-hubert-base install, users can directly use japanese-hubert-base installed effect in huggingface.co for debugging and trial. It also supports api for free installation.
japanese-hubert-base install url in huggingface.co: